Results 1 to 24 of 24

Thread: Graphic/Image/Modeling Benchmark

  1. #1
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts

    Graphic/Image/Modeling Benchmark

    Hi guys, some time ago I wanted to create a benchmark, but because I was unsure about some things I didn't continue it.
    Today I searched for some interesting files to test m1 and stumble over mesh, dds, tga. Because I started collecting some files of these sorts I thought to start a benchmark of exactly those types. I don't know the file size yet, but I think it will be between 400-800mb not higher.
    The plan is to have a mix between well known formats and less used formats. The focus will be games because there are those interesting formats like mentioned dds and tga. I will include also jpg for example to have a "complete list" of files, but only a little amount.
    I am at the beginning and think of the following types

    highes amount
    bmp (will find enough)
    tga (already found enough)
    dds (already found enough)
    mesh (already found enough)
    raw (found enough cr2 and nef)
    svg, svgz (found enough)
    tiff (didn't search yet, but don't think I have any on my pc)

    smaller amount
    jpg (will find enough)
    gif (will find enough)
    png (will find enough)
    (psd) (found enough)
    (3ds) (need some)
    some custom 3d-model-files (will find enough)
    (odg) (would need some)
    ico (will find enough)
    cdr (still need)
    xbm (still need)
    ttf (Fonts)

    I will complete this list and collect all file files. If you have ideas of file types that are missing / where to get them or anything else post it here.
    I won't investigate too much and won't try to create a perfect list, but it should be a good and interesting mix.

    EDIT1:
    Found enough psd Files

    EDIT2:
    Will add cdr, xbm and likely some fonts
    Found enough svg files

    EDIT3:
    Found enough raw files (nef and cr2)
    Last edited by Simon Berger; 8th April 2009 at 02:08.

  2. #2
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,612
    Thanks
    30
    Thanked 65 Times in 47 Posts
    Yeah, there's a lot missing. From my TC configuration:
    Graphics_SearchFor=\.(3cn|3d|3d2|3ds|3dml|555|565| a3d|abf|acr|ai|ai[345678]|abr|ask|aswcs|aswf|asws|atm|awd|b3d|bar|bitmap|bl end|bmi|bmp|bsz|bw|cals|cam|can|cdr(|[3456])|cdt|cel|csfdb|cgm|cit|clp|cmx|cot|cpi|cpt|cpt6|c r2|crw|csl|ct|cut|dc[msx]|dd[bs]|dia|dib|djv|djvu|dng|dpgraph|dpx|drw|dxthemepro|e cw|ef3|efx|emf|emz|eps|epsf|f96|fax|fcx|fits|flo|f lw|fnt|fon|fpx|fsh|ftf|fxd|fxm|fxr|fxs|fy|fyb|g3|g 3n|gem|gif|gml|graphml|graphmlz|gsf|gsm|ham|hpgl|h pgl2|hr[fz]|icb|icl|icn|icns|ico|icon|iff|imeshdef|img|iw4|iw 44|j2c|j2k|jbg|jbig|jbig2|jff|jfif|jif|jiff|jng|jp[2cef]|jpeg|jpg|jpg_t|jpm|jtf|kdc|kfx|koa|ksf|lbm|ldf|lj p|lwf|m2s|max|mdl|mh|miff|mi[lx]|mng|mnu|mrw|ms3d|msstyles|mvg|mvl|nef|ngg|niff|nl m|nol|nrf|oaz|ofm|ofx|orf|otb|otf|p3d|p7|palm|pan| pat|pbm|pc[2cdltx]|pcds|pdd|pef|pfb|pfn|pgm|pic|pict|pk|pn[gmt]|pntg|pob|ppm|ps[bdp]|pxr|pyx|q3d|qfn|qfx|qtif|ra[sw]|rgb|rgba|rib|ric|rl[ace]|s3d|sep|sda|sff|sfw|sgi|shx|sid|skin|skn|skp|st|s un|svf|svg|svgz|sw|sxd|targa|tct|tg4|tga|tgf|theme |thumb|tif|tiff|tt_|ttf|tub|udi|ufo|uis|uml|v3d|vd a|vfx|vsd|vst|wbmp|wmf|wmz|wpg|wsz|x3d|x3f|xbm|xcf |xdruid|xfx|xgml|xif|xmpskin|xpm|xwd|ygf|yuv|zbr|z eiss|zgm|zmf|zook|zuml)$

    :P

    But seriously, I think that you miss cdr, xb(p?)m.
    Last edited by m^2; 6th April 2009 at 19:12.

  3. #3
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    xbm looks interesting didn't know it.
    I will include both if I find them anywhere.

  4. #4
    Member m^2's Avatar
    Join Date
    Sep 2008
    Location
    Ślůnsk, PL
    Posts
    1,612
    Thanks
    30
    Thanked 65 Times in 47 Posts
    And maybe also some fonts?

  5. #5
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Hmmm yeah, I will think about it. Really wouldn't be out of place

  6. #6
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hi Simon!,
    My favorite benchmark is about games (I'm a former amateur game developer ). Nice to see you are about to make a new benchmark. Here is my comments about the list:
    - MD3 model and BSP map files (mainly used by line of Quake3 engine based games)
    - MD5Mesh model and CM, PROC etc map files (mainly used by line of Doom3/Quake4 engine based games)
    - Unreal Tournament maps
    - .MAP file format (mainly used by any game editor with small variants and this format is fed to the map compiler for producing final map file)
    BIT Archiver homepage: www.osmanturan.com

  7. #7
    Member chornobyl's Avatar
    Join Date
    May 2008
    Location
    ua/kiev
    Posts
    153
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I whant to criticize your hires list a bit, so
    bmp - not the best choise for uncompressed format because images written upturned, and there lot of padding and headers
    tga - pretty well for 24/8 bit data
    dds - sounds interesting, contains dxt compressed data widely used in computer games
    raw - interesting only in case if it contains actual nonpocessed sensor data from the camera, if not its almost equal to tga
    svg - vector graphic(which is good) written in xml form (which is not, because makes it closer to enwik)
    svgz - same thing compressed with gzip(say hello to precomp)
    tiff - widely used in polygraphy, interesting if contains CMYK/bitonal/16bpp data and uncompressed, RGB24 makes it equal to tga, use of compression turns image benchmark into something totally different

  8. #8
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Thank you. Those constructive critic is what I wanted to see!
    You seem to have much knowledge in this area. Far more then I have.

    bmp - not the best choise for uncompressed format because images written upturned, and there lot of padding and headers
    You might be right here. But I want to create a real benchmark and bmp is still a very important format. I know mainly png and tga gets place of it.

    > tga, dds
    Yeah, those two will be most important

    raw - interesting only in case if it contains actual nonpocessed sensor data from the camera, if not its almost equal to tga
    Right, I try to find camera images for this format. Found raw-Files also in a game but have to investigate if this really is the image format or a custom one.

    svg - vector graphic(which is good) written in xml form (which is not, because makes it closer to enwik)
    Yes thats a problem I have reached too. Svg is really important so I don't know what to do yet.
    I will have a look if it could be precompressed or if there is a possibility to create a special model where not only good text compression will win. If thats the case I will add it. If not I maybe really decide to leave it out.
    Svgz: Yes I know . Thats one more reason to include zlib recompression. osmanturan will like it if he really will add it to Bit. If I won't add svg's I absolutely will add some svgz's.

    tiff - widely used in polygraphy, interesting if contains CMYK/bitonal/16bpp data and uncompressed, RGB24 makes it equal to tga, use of compression turns image benchmark into something totally different
    image data types are closely linked to compression. Thats a media benchmark adding files like tiff, png and jpg is important. At least because there are many preprocessors and special models out now. But now worry. I won't add too many compressed files that those compressors won't win only because of compressed files.
    Thanks for warning. I didn't know that tiff includes compression. I will have a look if it is always compressed or only has the possibility
    Last edited by Simon Berger; 6th April 2009 at 21:29.

  9. #9
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Simon Berger View Post
    ...Svgz: Yes I know . Thats one more reason to include zlib recompression. osmanturan will like it if he really will add it to Bit...
    Thanks a lot. But, still can't find a good way to store recursively processed blocks
    BIT Archiver homepage: www.osmanturan.com

  10. #10
    Member
    Join Date
    May 2008
    Location
    Kuwait
    Posts
    290
    Thanks
    21
    Thanked 16 Times in 12 Posts
    Quote Originally Posted by Simon Berger View Post
    Hi guys, some time ago I wanted to create a benchmark, but because I was unsure about some things I didn't continue it.
    Today I searched for some interesting files to test m1 and stumble over mesh, dds, tga. Because I started collecting some files of these sorts I thought to start a benchmark of exactly those types. I don't know the file size yet, but I think it will be between 400-800mb not higher.
    The plan is to have a mix between well known formats and less used formats. The focus will be games because there are those interesting formats like mentioned dds and tga. I will include also jpg for example to have a "complete list" of files, but only a little amount.
    I am at the beginning and think of the following types

    highes amount
    bmp (will find enough)
    tga (already found enough)
    dds (already found enough)
    mesh (already found enough)
    raw (found some but need more/some others) (http://www.rawzor.com/developers/)
    svg, svgz (found enough)
    tiff (didn't search yet, but don't think I have any on my pc) (http://www.fileformat.info/format/tiff/sample/index.htm)

    smaller amount
    jpg (will find enough)
    gif (will find enough)
    png (will find enough)
    (psd) (found enough)
    (3ds) (need some)
    some custom 3d-model-files (will find enough)
    (odg) (would need some)
    ico (will find enough)
    cdr (still need) (http://product.corel.com/en/draw/10/...o_download.htm)
    xbm (still need) (http://www.fileformat.info/format/xbm/sample/index.htm)
    ttf (Fonts)

    I will complete this list and collect all file files. If you have ideas of file types that are missing / where to get them or anything else post it here.
    I won't investigate too much and won't try to create a perfect list, but it should be a good and interesting mix.

    EDIT1:
    Found enough psd Files

    EDIT2:
    Will add cdr, xbm and likely some fonts
    Found enough svg files
    and you can refer to Precomp 0.38 for other formats:
    sketchup *.skp
    Autocad *.DXF, *.DWG
    Solidworks *.SLD http://www.structureworks.net/SWmodel.html
    basicly refer to this site http://www.fileformat.info/format/all.htm it would save time

  11. #11
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    As mentioned before I won't use many do a benchmark with many compressed files.
    Also I don't want to use too many 3d and technical stuff. I thought about a AutoCad section but I guess I won't include it. Highest part will be real images/texture stuff.

    Another question is how to do the tests. I collect the files in folders for data types at the moment. Thats good for archivers that don't sort the files. Because of this reason I consider to change it and mix the types to give them an advantage.
    A second idea was to leave them in an own folder, compress every folder seperately and count the results together. Then we have the chance to see where a compressor loses ground. I would do such a testing only if I have a really good test-platform. I have to build something because I only do every small things at the moment by hand.
    Last edited by Simon Berger; 7th April 2009 at 14:47.

  12. #12
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I prefer second idea (compressing the folders which have set of single file type). Btw, what do you think about "single file compressor"? I hope, you won't use TAR files
    BIT Archiver homepage: www.osmanturan.com

  13. #13
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Sure . I am going to use archiver functions wherever I can but for single file compressors using tar files. Because of this I would prefer mixing the files to give an advantage to real world archivers.
    What are your arguments against tar and what would you use?
    Compressing file by file would hurt lz based compressors too much and also most compressors don't store the filename. CM compressors propably win because of single file compression with the addition of no filename storage...

  14. #14
    Programmer osmanturan's Avatar
    Join Date
    May 2008
    Location
    Mersin, Turkiye
    Posts
    651
    Thanks
    0
    Thanked 0 Times in 0 Posts
    IMO, there are two ways to give approximately equal chance to "archivers" vs "single file compressors":

    1-Divide the chart into "archivers" and "single file compressors" like many of us offered to Nania many times (finally he made it ). Archivers should be fed by whole folders and compressors should be fed by each files.

    2-SqueezeChart's Way: You can group similar files together (ideally an entropy measurement should be used but, it's not necessary) into set of TAR files. And you could feed these TARs to compressors.

    Each of the ways have their own advantages and disadvantages. For example, if you use TAR files for CMM4, it can't use x86 filter at all while CCMX can use it. I think, first option makes sense but, it's maintenance is a bit troublesome.
    BIT Archiver homepage: www.osmanturan.com

  15. #15
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    In my opinion you see it from the wrong point of view.
    If ccmx can handle several filter and model inside one file thats a very nice feature. I don't want to create a benchmark-platform where every compressor has the same ratio.
    If someone like toffer would change/improve his compressor because of such a benchmark I would be happy... .
    I got your point but why should I do single file compression that cm based compressors have an advantage and lz- based no chance.
    On the other hand I don't have a decision yet.

  16. #16
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    Simon, please pay high attention to TIFF files. TIFF is actualy the container format and very flexible one. It can use different color models, i.e. GreyScale, Half-tone, RGB, CMYK, YCbCr, CIE Lab. It supports 8, 16, 32 and 64 bits per channel in integer form and even 32 and 64 bits per channel in floating-point. Furthemore it can use different types of compression - RLE, LZW, LZ77, ZIP, H.264, JBIG, JPEG and CCITT Group 3\4. So for example some TIFF file can be roughly speaking a JPEG file.

    Also pay attention to DDS files too. They can be compressed or uncompressed and be one of the DXT1\DXT2\DXT3\DXT4\DXT5 types.

    Hope it will help a little bit.
    Last edited by Skymmer; 8th April 2009 at 04:12. Reason: 'Color spaces' phrase changed to 'Color models'

  17. #17
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    If you read my posts you see that tga and dds are most important for me.
    Would be nice if you could point me to some tiff files. Where to get them. I most times have no problem but I don't want to investigate too much time finding out about a mix of several specifications, compression and so on.

  18. #18
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    Quote Originally Posted by Simon Berger View Post
    If you read my posts you see that tga and dds are most important for me.
    Oh, Simon, I suppose I was unclear in my reply. I didn't wanted to say: "Please test more TIFF files !" I wanted to say that I you're gonna test TIFF files then check them thoroughly or it suddenly may end up as testing of already compressed data. I believe XnView tool can help you. Its just great freeware graphic viewer. By calling file properties (ALT+Enter) you can see what kind of TIFF (or any other file) you have.

    Quote Originally Posted by Simon Berger View Post
    Would be nice if you could point me to some tiff files. Where to get them. I most times have no problem but I don't want to investigate too much time finding out about a mix of several specifications, compression and so on.
    Well, there is no much help from me here. I don't use TIFF files too much in my life but I can ask a friend of mine which is involved in graphic advertisement production if you're not rushing. Few days ago I took the 300 MB TIFF file from him to see how much I can shrink it down. Maybe he knows some usefull links. It will be tommorow, well today already

  19. #19
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Ok. I got this warning earlier

  20. #20
    Expert
    Matt Mahoney's Avatar
    Join Date
    May 2008
    Location
    Melbourne, Florida, USA
    Posts
    3,253
    Thanks
    304
    Thanked 768 Times in 482 Posts
    Sorry if I don't see the point in testing over hundreds of obscure image formats If you want to encourage research in the best image compression algorithms, use a standard format like bmp. It is more important to select representative images than formats, and use stuff that hasn't been converted from jpeg or something lossy. The Kodak set is probably good, and there are already published results for it. rafale.bmp is a bad choice because the low bits have been removed. It is actually 16 bpp, not 24.

  21. #21
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    As written before I concentrate on formats used in games like dds and tga. Do you really think those files will be compressed the same like bmp?
    Surely I will select native - non converted files no worries.
    It's interesting for me and thats the argument to do it. Releasing it to public is only an addition.
    I am not interested in benchmarks like enwik8 or any single file compression because you won't do it with the program later. No problem to do it because it is important to test the own compressor while developing.
    For this reason I will create some situations of usage.

    1) Releasing ratio
    Here it will be a result of compressed size and decompression time. I think I will create a somewhat dynamic formular choosing an archiver with a reasonable time and giving any higher times a penalty that will grow non linear. But I will decide this if I have a look at the first results.

    2) Backuping ratio
    compressed size + compression time. For the compression time it will be the same then written for Releasing ratio

    3) Sending ratio
    Bad name yet but I didn't want to break one word serie. It will be a mix between all three and means if I compress something for later usage or to send to a friend. It will be a formular even to the one from maximumcompression or mixing 1 and 2 together.

    Maximum compression will be still important. I won't forget cm compressors. Maybe I will also give penalties also for size.

  22. #22
    Member chornobyl's Avatar
    Join Date
    May 2008
    Location
    ua/kiev
    Posts
    153
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Skymmer
    Furthemore it can use different types of compression - RLE, LZW, LZ77, ZIP, H.264, JBIG, JPEG and CCITT Group 3\4.
    I heard about everything else, but H.264 is just too much
    Quote Originally Posted by Skymmer
    I believe XnView tool can help you
    Also IrfanView, it shows more info to me
    Quote Originally Posted by Simon Berger
    Surely I will select native - non converted files no worries.
    But feel free to convert one lossless format into another like bmp/png/tiff => tga (if there's no color conversions of course)

  23. #23
    Member Skymmer's Avatar
    Join Date
    Mar 2009
    Location
    Russia
    Posts
    681
    Thanks
    37
    Thanked 168 Times in 84 Posts
    Simon, if you still interested in some TIFF files I've uploaded a couple of scans in TIFF uncompressed format. Its the scan of "Autechre - Basscad EP (Basscadetmxs)". Well, as I promised I've asked my friend ages ago but seems he doesn't know some usefull links and don't want to help by providing his files.
    Anyway, I hope it will help somehow
    Ahh, and the LINK of course. Let me know if you have troubles downloading it.

  24. #24
    Member
    Join Date
    Oct 2007
    Location
    Germany, Hamburg
    Posts
    407
    Thanks
    0
    Thanked 5 Times in 5 Posts
    Thank you. I am still interestend in files and the benchmark . I have some other projects at the moment but I collect files here and there and at any time I will do it.

Similar Threads

  1. UCI Image Compression
    By maadjordan in forum Data Compression
    Replies: 5
    Last Post: 19th August 2017, 23:15
  2. Logistic mixing & modeling
    By toffer in forum Data Compression
    Replies: 44
    Last Post: 3rd March 2011, 06:25
  3. GraLIC - new lossless image compressor
    By Alexander Rhatushnyak in forum Data Compression
    Replies: 17
    Last Post: 29th November 2010, 21:27
  4. Image retargeting
    By Black_Fox1 in forum Forum Archive
    Replies: 1
    Last Post: 24th August 2007, 04:02
  5. image compressors
    By maadjordan in forum Forum Archive
    Replies: 5
    Last Post: 13th August 2007, 09:28

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •