What are the current MOS data sets that are used in image compression domain ?.The methodology of the MOS tests is also very important as there are a lot of variables at play here.In my humble opinion factors influencing the experiment are as follows:
1)Equipment and environment
Screen used and the distance of the test subject , the lightning conditions as well as how well trained the test subjects are to detect compression artifacts in images.
2)How diverse are the images in the dataset ?
There are a lots of different types of images in the world namely photography(aerial , portrait , scenic ), black / white , medical images etc.The image dataset must represent the diversity in the world.
3)Defining the relationship - (bpp -> MOS score) for each image.
There are different standards for quality depending on the use cases such as psychotically lossless , "Good" , "Bad" )
4) Source images
I believe each source image should have the following variants each with their own MOS score when applicable
- The RAW image (the raw image is the source of truth and contain minimal bias).Moreover raw images can be used to generate other images with different settings (i.e light balance, special effects etc...)
- The JPEG image generated by the camera.
- The RAW image enhance by a photographer saved in a lossless format.
An efficient subjective image quality assessment can help to build objective quality metrics of the future , image formats as well as help research as a whole given the high costs associated with subjective image assessment.
I wonder what are the views of the compression experts , what factors i have missed out and how it can improved ?