A Video Player module, a new version of HECTOR, and the endless stuggle with edge cases

29 August 2017

It's been a long day today working out a solution to an issue I was facing using a third-party library. One of the components I'd created under the 'Media' modules was a basic Video Player which was working brilliantly until I came to actually announce the public beta. To keep things neat and tidy, I wanted to package each HECTOR module up into a single file which contains everything needed to run the module, including external DLLs. I felt I'd worked out a way to do this really well, but I came unstuck with a third-party library that the VideoPlayer module used - Emgu CV.

There is a short version of this story...

I've uploaded version 0.2 public beta of HECTOR today, and I have a working Video Player module, bringing my total module count to 39! I recommend ditching any earlier version of HECTOR that you may have, because this new version changes the module loading structure somewhat.

Now if anyone is really interested in the long version

Emgu CV seems like a really nice library - it basically acts as a C# wrapper library for OpenCV, with some other pretty neat features added in. I'd also found it to be the only .NET-compatible OpenCV library which actually reliably worked. Using NuGet via Visual Studio, there are a whole bunch of OpenCV libraries that promise to work with a Visual Studio project, but I found that the majority would not install, complaining of issues between the .NET version I was targeting (4.52) and the version required by the library. I think a lot of this - although I may be mistaken as I didn't spend a long time debugging it - relates to the fact that many OpenCV wrapper libraries have been started over the years, were probably built for a specific project, and have then been effectively abandoned.

Anyway, I found Emgu CV pretty quickly after a few failed attempts at other OpenCV libraries, and then got stuck into using it for my Video Player module. Of course, there are plenty of ways I could have built a video player module (even using my own Quick Sync library for certain codecs), but I was keen to find an OpenCV solution for further use with other more complex modules later on.

Then the headaches start

After uploading the HECTOR public beta and the Media modules, I suddenly found that the packaged version of 'Media' wouldn't load into HECTOR. It took quite a while to track it down, but essentially Emgu CV loads a bunch of DLLs when it is first initialized, and it has an internal list of these DLLs. It expects to find them in an 'x86' or 'x64' directory and if it can't find them, it causes an exception. Technically, the Windows subsystem should be capable of finding the DLLs automatically if I installed them into the System32 folder or if I p/invoked AddDllDirectory and put the DLLs in a specific folder, but I found that Emgu CV *still* fires an exception. In the end, I cleared this exception by adding an empty 'x64' folder to my application's folder - Emgu CV seems to be firing an exception if it can't find the folder, not realising that the DLLs can be loaded elsewhere.

The headaches could also be that I do still seem to have the flu

This really highlighted a failing in my HECTOR module approach, in that while I'd successfully managed to cater for DLLs that the HECTOR module calls, I hadn't really catered for the fact that a HECTOR module may call a DLL, which in turn tries to load another external DLL and does it in a way that I hadn't expected. So, what I needed to do was try to cater for these edge cases.

I decided that I didn't like the approach of adding a blank 'x64' folder into my main HECTOR path, it just seems unnecessary and not very tidy. I also felt that Emgu CV's approach of looking for the DLLs only in specific folders and throwing an exception if they're not there is a grey area in terms of best practice - yes, you should throw an error if you can't find the DLLs, but I would prefer it if you didn't throw an error if the DLLs can still be loaded by the Windows subsystem when they are placed in an unexpected location. I believe the loading mechanism could be made more resilient and I'm hoping I can contribute something to the Emgu team when I've given it sufficient thought.

In the meantime, I've forked the Emgu CV code and have added my own method which can be called to manually initialize the Emgu CV library with a different folder for the DLLs. I then created a way of flagging certain files within the module packages as 'external', meaning they need to be unpacked and stored on-disk before the module can run. The running module can then find out where the external files have been unpacked to, so that this information can be used to help Emgu CV (or any other external library) find the files it needs.

I'll need to put my fork of Emgu CV online very soon, but I'm going to wait a couple of days just to make sure I've really thought this through in the best way possible. I think the code I've written into Emgu CV can probably be neatened up a bit more so that Emgu CV's standard method still works and my solution can just sit 'on-top', or even act as an automatic fall-back when the standard method fails.

What about supporting edge cases?

I realise this approach doesn't actually support Emgu CV's way of doing things (because I've had to edit their code), and therefore I'm not really supporting them as an edge case. It's disappointing in some senses that I can't support this scenario, but then again Emgu CV is the one throwing the error even though HECTOR is making sure it can load the DLLs. So, what started out as me trying to support Emgu CV as an edge case has ended with me modifying the Emgu CV code to support me as an edge case instead. I don't think this is a bad thing - what I'm supporting instead is really a way for a programmer to write a module and get third party libraries like Emgu CV working with a minimal amount of code revisions, and also I hope that those code revisions actually improve the third-party library. I think that's as good as I can really get it.