I know it isn't pretty but it doesn't worry me either. It's still a part of the engine with pretty much no circulation of oil just a few splashes onto the hot metal, not really an ideal setup for keeping it clean. On previous Volvos with the same in line five but without the splash plate I've also had oil deposits on the lip below the lid while cams and everything have been perfectly clean.
You could argue that this is as Volvo meant it to be since it's been serviced at their dealership a few miles from the Gothenburg factory for its entire life before I bought it. It has gotten slightly cleaner since I started doing oil changes every 10000 rather than 20000 km though.