ThumbView Postmortem

It is recommended that you check the Modules page first, to get a basic understanding of the different modules.

Introduction

When making our game we had troubles with our TGA and PCX files, because new versions of images would be passed around together with old versions. Images would be split up, get backgrounds removed to be used as several layers, etc. And each time we wanted to check 'what was this image again?' we had to start some image program that takes a while to load. (Bad image management is not the issue here folks.) But we could see some images (like BMP) as thumbnails. This was a bit irritating and I started to look for some program that could show thumbnails of TGA/PCX images in Windows Explorer. I found one, but it had a price (a very small one, but still a price). I thought to myself, “it can’t be that hard”. This planted the seed of this project. A summer passed and I decided to start learning some libraries. First I had never touched ATL nor COM, and neither had I used .NET nor C#. I hadn't even used .DLL's. The seed took root. I don't really remember how I came across it, but the "IExtractImage interface is used for generating thumbnails in Windows... all I have to do is implement that!". The seed started to grow. The 'proof of concept' took one evening to finish, it displayed the thumbnail of a renamed .bmp (renamed to .wtf) using Win32 API for loading the image. Then my friend added some water to my ThumbView plant. He had made the bitmap loading routines (dubbed 'Beermap') used in our game to load the TGA and PCX images. After discussing with people on #gamedev (on Afternet, see Credits) I started to design. I wanted a pluggable system for image types, because:

After that was a long journey of coding, refactoring, fixing bugs, redesigning, fixing new bugs and so on and so forth. But now the plant has grown into a nice little Bonsai tree. Let's see what branches that worked and what I should've cut.

What went right

Making a 'proof of concept'

I started by looking at tutorials and searching through MSDN. I didn't decide on much of ThumbView until I could read regular .bmp's and get their thumbnails but discussed as another format, like .wtf. Of course, I did have the occasional brainstorms but I didn't write anything down in stone. This took me one night and proved that I could make thumbnails of whatever file extension I wanted. Then I used a library I knew worked and knew how to use, the image loading library from our game (Beermap) to make TGA and PCX images work. Now started the actual brainstorming and how I would make the file support dynamic. Thumbnail Extractors were born.

Thumbnail Extractors

By removing all the actual image extraction code from the ThumbView core and placing those in external DLL's gave ThumbView a nice flexible touch. New image types can be plugged in without the user having to reinstall a new version of ThumbView. I can also make new Thumbnail Extractors wrapping other libraries, or add support for say 3D models to be rendered as a wire frame model. ThumbView handles the windows registration and calls your Thumbnail Extractor; you use the imagination for the rest.

SourceForge

Having a safe place to backup all your files is not just a good practice, it's also very convenient. Using CVS has been a habit since my first big game (that never got finished, but was a horror to keep track of versions). So I needed to have a CVS server. I almost thought of installing one on my old computer, but I know I would never backup on a regular basis. Plus I needed some web space for the homepage. I started with my ISP provided web space, but they didn't support PHP. Nor did my GDNet+ account on GameDev. I have used SourceForge for larger projects, but they don't have a limit on how big or small the project must be. So I registered another project. This has worked like a charm. Both CVS and a web space with a simple URL, that was all I needed and it’s free. What more could I ask?

Refactoring

This was the first project I refactored every chance I got. I never ever copy pasted code; as soon as I saw myself doing it I refactored the code into a separate function. Those functions often got used a third time, and when finding a bug it would get fixed at three times at once. This would work out great most of the time, but sometimes a bug would get introduced by the new function, although it would be a simple error and fixed almost immediately. 'Refactoring mercilessly' is an Extreme Programming idiom, and I liked it.

When making the .NET version of ExtractorManager some functionality creped into the GUI because .NET was so easy to use when manipulating files. But when the requests for a non .NET version of ExtractorManager came up I remade the console version, refactoring and reusing everything I could from the .NET version. At first it introduced some bugs but they were quickly solved. Now I can make a wxWidgets version or Qt version or whatever version of ExtractorManager without needing to do more than the actual user interface.

The introduced bugs were dumb mistakes, showing that I need to do this more often to get a hang of it.

Making DLL proxy classes

Making a simple library (DLL) wrapper (called LibraryWrapper) was one of the best wrappers I did, because it took 5 minutes to write (encapsulating Win32API calls to load, free and use templates for getting process addresses). It halved all code where I connected to DLLs. Although it's worthless now that I have discovered the use of LIB’s when working with DLL’s it's still used in the ThumbView code.

From time to time I have to connect to Thumbnail Extractors. So I made a proxy class inspired by the LibraryWrapper, namely ExtractorProxy. By just taking the file path in the constructor I could use the ExtractorProxy just like if I was working directly with the Thumbnail Extractor. That made my code both smaller and easier to read.

Trying external libraries

From a learning perspective this was a great thing. I learned how to use ATL with COM, use C# with .NET, and learned to use DevIL. Since I'm a student making this project on my free time this was a great learning experience and using external libraries is something I will do more often since that let’s me concentrate on my own project. I especially enjoyed .NET and will never work with MFC again, unless I get paid to of course.

Listening to users

I think this is the most important part of any project. And it was a very positive experience. Several users wrote mails or wrote in forums giving suggestions on how they felt that ThumbView should work. I also got several tips from friends. One tip was so great that it’s my next project. Bug reports were also very useful. Having a thread on GameDev worked well because anyone could post their comments while I at the same time was advertising ThumbView.

Making it user friendly

Taking the time to figure out how to register my own file type with windows so that users can just double click on a new Thumbnail Extractor and it'll install itself was all worth it. Because it's so simple to install Thumbnail Extractors, everyone that has downloaded ThumbView has downloaded several Thumbnail Extractors too. And taking the time to make a simple user interface (ExtractorManager) for managing what Thumbnail Extractors are installed and which ones are enabled was also worth it.

ThumbView was meant for developers, i.e. programmers. Because they could make their own formats and have thumbnail support for them. After a while it hit me, that in this case game developers are also users. Also, artists might want to be able to see their works as thumbnails in explorer. Everyone just wants to do their stuff (work on their game) so they just want something small that works. That's what I would want, a light (Lite) version. So that's what I did. I wrapped the most capable Thumbnail Extractor and compiled it together with ThumbView, while at the same time removing all the extra stuff like the GUI's. This was worth it because the download count for the regular and Lite versions are the same, meaning it was appreciated.

Using DebugView

If it wasn't for this great tool I may never had finished ThumbView. I was missing a good logging system in ThumbView. So I used OutputDebugString() like most people do. The problem was that those messages were picked up by Visual Studio, so I couldn't see them when Windows was calling my DLL (i.e. I wasn't debugging). DebugView picks up these messages! So I could have debug information in the same time I was running ThumbView as a normal user.

What went wrong

Using DebugView

Using DebugView was great when I was debugging, but what about when stuff went wrong on users computers? Asking them to download DebugView and sending me the info would most likely not work, so I haven’t even tried. I should have invested some time in writing a file logging system for errors. A simple file with all the needed info that the user can just send to me if they experience problems would be a nice solution.

Using DevIL

When I decided to use DevIL for a Thumbnail Extractor pack, I didn't want to have to make the user download the three needed DLL files. As DevIL is an open source project I (naively) thought that I could compile it into LIB files instead and statically link to them. This took over one whole week of desperately trying to solve linker errors. I finally made it work when compiling to a DLL (as the DevIL pack) but couldn't get it to work when making a LIB for ThumbView Lite, so that's why ThumbView Lite comes with 3 extra DLL's. This also means that if DevIL ever updates the DDS or TIF support, I will have to travel through linker-error-hell once more to release a new DevIL pack. Compared to ThumbView Lite where if the user has any problems with the actual generated thumbnail image (i.e. a problem on the Thumbnail Extractor part and not on ThumbView) he/she can just go over to the DevIL homepage and download the latest DLL build. I should have just used the DevIL like it was made to be used, using the DLL’s.

DLL's

This was my first project ever touching DLL's. Although they are useful, and allows for patching/updates I have found that it's best for 3rd party libraries (like DevIL). I did have to make the registry operations as a DLL since I was using C#.NET for the user interface (ExtractorManager), and those functions were written in C. This wouldn't have been such a problem if I only had understood the concept of DLL/LIB (other type of LIB). Where the actual implementations are in the DLL but by importing the LIB you can use the functions without explicitly loading the DLL. At least 2 full weeks went into the DLL handling code when it comes to Registry.dll and Thumbnail Extractors, but most of it became useful reusable DLL handling code later used in a Thumbnail Extractor proxy class. I should have planned this part better, but lack of experience hindered me. I should've used p/Invoke in C# (as I did) but use the Registry.lib for the other parts of the program so I wouldn't have to write wrapper classes for loading Registry.

Before I made the GUI in C#.NET I had decided that the Registry should be a DLL. Because then I could fix bugs in the Registry module and the user could just download a small patch instead of the reinstalling everything. I would only have to recompile one module, instead of all modules using Registry. This was a good thought. But since this was a small project with only me as the coder and a recompile of the whole project takes less than one minute, I'm not really saving any time. I had an idea for auto-updating, but SourceForge didn't approve of it, so another reason for separating Registry was flushed. I also decided not to patch that often, but rather have extensive testing before releasing a version. And I would only release a new version when several bugs had been fixed, because I personally hate applications that patch all the time, they seem so broken and it's just a pain when there's no auto-update. Meaning I would never release just a new Registry.dll. “Luckily”, because I did the GUI in C#.NET I had no choice but to have Registry as a DLL, meaning it became the right choice.

Another problem that rose was that the code in Registry.dll is needed in ShellExtension.dll but since ShellExtension is called by windows, the working directory gets set to C:\Windows\ and not the directory where ShellExtension is (which is the same directory where Registry is). I could have solved this in various ways. Writing the Registry.dll directory path to the windows registry on install and have ShellExtension check the registry. Or compiling the Registry as a LIB for ShellExtension and as a DLL for the C# version of ExtractorManager (but at the time I didn't know of the connection between DLL/LIB). But I choose the simplest way that I knew of, I got the actual directory of ShellExtension.dll, removed the filename and added “Registry.dll”. Now I would've chosen method number 2, making one compile for ShellExtension and one for ExtractorManager.

I over engineered the use of DLL's as modules, and worse: had reason to thanks to the use of C#.NET (read below).

Combining languages

Although it seemed like a cool and good idea at the time, making the GUI in another language wasn't that great. Although whipping up the actual interface was a breeze with .NET, I still had to be able to access not only Registry but also the Thumbnail Extractors. This meant that I had to make a bridge DLL so I could load Thumbnail Extractors dynamically. See, the way p/Invoke works you have to know the DLL name on compile-time. Not very pluggable now is it? For convenience I just inserted some proxy functions into Registry, which actually breaks Registry as the 'windows registry interface' for registering/unregistering Thumbnail Extractors. I don't know how it is with managed C++ working with unmanaged C++, but I'm guessing it would have been a lot simpler. If I hadn't pulled C#.NET into the picture I could've ended up much closer to the ThumbView Lite version, i.e. one DLL for the actual shell extension but with all the functionality of the full version.

I can see the worth of mixing languages. Like a GUI in C++ and wxWidgets but Python to make the functionality. But in this case it was a bad choice that caused the project to bloat adding twice the development time needed. The time won by the graphical interface in .NET should be taken into consideration though, since I think making it in wxWidgets or something else would have taken three times longer. But still, the overall project time would have been a lot shorter not to mention the amount of code kept in the end.

One thing that surprised me though, because my target is game developers and not common users, many people don't have the .NET Framework installed! Even though the .NET framework comes with the SP2 update. The only reasons I got was "it's evil", well I won't discuss what I think of that statement here. So now people couldn't use my shell extension because the user interface was using .NET. Instead of concentrating on making ThumbView Lite, I made a console version of the ExtractorManager. This took some time because of the refactoring of the functional code and the long testing. Not to mention making a installer that could detect .NET and alert the user in a way the user would feel comfortable with. I should just have ignored those few requests and made ThumbView Lite directly, since it ended up not needing .NET anyway.

Managing Thumbnail Extractors

This turned out to be the most time consuming part of ThumbView, mostly because of the hassle with the Windows Registry at runtime. At the same time it was a fundamental part of what I wanted ThumbView to be like. I wanted dynamic plug-in capability of Thumbnail Extractors. A major error I did with DevIL was to try to statically link the code into my DevIL Pack Thumbnail Extractor, but I felt I didn't have a choice. The easy pluggable part would be broken if I would have to make the user download a separate package for the DevIL DLL's. Now when DevIL Pack is stable, the only thing that can be updated is the actual image support. I.e. unless they add new formats, it would be easier for me if the user could just download the latest DevIL DLL's from the DevIL homepage. This is the approach I choose for ThumbView Lite. If DevIL updates it support or adds some new file types, I will make a simple installer that installs the DevIL Pack Thumbnail Extractor and the DevIL DLL's. If I only had done this before I would have saved over one week of linker error hell, and updating would be so much simpler.

Although I'm not sure on the user side, do they appreciate, don't understand, or ignore the fact that image support in ThumbView Lite is in the hands of DevIL? Currently, no one has said a thing about that (DevIL hasn't been updated in quite a while). So only time will tell or maybe it won’t.

How I would do it today

Ah yes, my favorite part. What have I learned from this project, and how would I do it differently if I ever was going to do it again? Well I could either change what I have or start from scratch, here are my thoughts.

Changing the existing code

I would still keep the two editions, the full and the Lite version of ThumbView. But I would not make two versions of the user interface (ExtractorManager). Although I did love the simple .NET, C# made me separate my code in to many modules because it wants everything to be “managed” (i.e. managed C++ or C#). So by making a user interface in wxWidgets I could compile Registry into where it's needed, thus only releasing an executable GUI (ExtractorManager) and the actual shell extension (ShellExtension). I would do this by simply ripping out the C#.NET ExtractorManager, and supply the console version until I get enough motivational e-mails and/or donations to make the wxWidgets version.

Starting from scratch

There are a lot of registry operations in the code, that's necessary because of the plug-in system of Thumbnail Extractors. And a few file operations. Not to mention the GUI. All this makes this a perfect .NET project. I wouldn't have to write any Win32 API wrappers and could do all the things I need with simple commands. Meaning the development time would be cut down immensely. The only hinder is the actual Thumbnail Extractors. Using C# with unmanaged C++ code was a bit to disturbing. But managed C++ is supposed to be simple to use with unmanaged C++. That's why I would go with managed C++ .NET. The Thumbnail Extractors would probably be made exactly like they are now. Five simple C functions. I might try a class/interface approach if importing that from a Thumbnail Extractor DLL would prove to be simple, but my intuition says “no, it won't be simple”.

Only making ThumbView Lite could be done in a matter of hours using regular C++ and wrapping some image library, like DevIL. Although I still would use managed C++, so that I could reuse the code from the regular ThumbView.

Conclusion

This was not only a working and useful tool. It was also my FIRST FINISHED project! I have learned tons of stuff, and especially used several libraries. I think I have reached my goal. But it was thanks to all the wonderful people who encouraged me and helped me when I got stuck. Love to all GameDev people and to all my friends, especially my mentor and soul brother Torbjörn 'DrunkenCoder' Gyllebring and of course; my parents. :)

About Me

I'm a 23 year old crazy guy that stalks the intarweb! No but seriously. I have a Bachelors Degree in Computer Science - Real time Systems from The Royal University of Technology in Sweden. I'm currently studying for my Masters Degree in Computer Science - Human-Computer Interaction. But I prefer writing small applications like this. I'm currently working on a 2D action game with some friends, and I'm going to start another game developer utility soon. Besides coding I work out regularly several times a week, and have a great interest in movies and anime which I watch together with my friends.

  • Hosted by SourceForge.net
  • Support This Project
Valid XHTML 1.1 Valid CSS 2.1