News

Keeping track of HECTOR developments with Trello

16 November 2017

Up until now, the entire HECTOR project has been planned and designed solely in my head without putting pen to paper or making notes, other than in my own code. This might seem like an odd strategy, but to be honest it was a decision I made deliberately. My reasoning was, seeing as this was an experimental piece of software that existed purely as an idea that needed testing in the real-world, it was hard to really sit down and define its scope. In fact, I wanted to do the opposite, and not constrain the development at an early stage.

Secondly, I just saw it as a challenge - wondering how far I could get by planning it in my head.

Now the code is starting to mature and I can see a real solution emerging, I think it's time to put my ideas down, to show where I expect HECTOR to go from here and to start tracking progress. So, I've started gathering my thoughts together on a trello board, which can be publicly accessed below.

Using HECTOR to color-grade with LUTs

06 November 2017

A fun little project I completed on Friday was to get HECTOR running with standard Look-Up Tables commonly used in applications like DaVinci Resolve and Adobe Premier Pro to allow quick and efficient colour-grading of video signals.

This worked out fairly well, and showed how quickly it is with HECTOR to prototype and complete a video filter effect. I started the work late on 2nd November and had it finished on the 3rd, and I was working on other projects at the same time.

Here is a direct link to the module: http://hector.direct/modules/ColorGrader.LUT.html

I'm most likely going to open source the code for this once I've tidied it up a bit, so I'll leave the huge explanation of how it all works until then, but the basics of it is relatively simple. The system uses OpenGL shader language to perform the look-up, so it's quick and works well with live HD video footage. I haven't tried it with 4K yet but I see no reason for it not to work.

Rocketstock.com have a pack of 35 free LUTs you can use to test this with. I've found some LUTs from other sources don't work yet - it seems like I'm having trouble reading all the different CUBE files that are out there. If anyone tests this out and finds a LUT that doesn't work, please send it over to me as it'll help me with debugging.

A fun little project I completed on Friday was to get HECTOR running with standard Look-Up Tables commonly used in applications like DaVinci Resolve and Adobe Premier Pro to allow quick and efficient colour-grading of video signals.

A Video Player module, a new version of HECTOR, and the endless stuggle with edge cases

29 August 2017

It's been a long day today working out a solution to an issue I was facing using a third-party library. One of the components I'd created under the 'Media' modules was a basic Video Player which was working brilliantly until I came to actually announce the public beta. To keep things neat and tidy, I wanted to package each HECTOR module up into a single file which contains everything needed to run the module, including external DLLs. I felt I'd worked out a way to do this really well, but I came unstuck with a third-party library that the VideoPlayer module used - Emgu CV.

There is a short version of this story...

I've uploaded version 0.2 public beta of HECTOR today, and I have a working Video Player module, bringing my total module count to 39! I recommend ditching any earlier version of HECTOR that you may have, because this new version changes the module loading structure somewhat.

Now if anyone is really interested in the long version

Emgu CV seems like a really nice library - it basically acts as a C# wrapper library for OpenCV, with some other pretty neat features added in. I'd also found it to be the only .NET-compatible OpenCV library which actually reliably worked. Using NuGet via Visual Studio, there are a whole bunch of OpenCV libraries that promise to work with a Visual Studio project, but I found that the majority would not install, complaining of issues between the .NET version I was targeting (4.52) and the version required by the library. I think a lot of this - although I may be mistaken as I didn't spend a long time debugging it - relates to the fact that many OpenCV wrapper libraries have been started over the years, were probably built for a specific project, and have then been effectively abandoned.

Anyway, I found Emgu CV pretty quickly after a few failed attempts at other OpenCV libraries, and then got stuck into using it for my Video Player module. Of course, there are plenty of ways I could have built a video player module (even using my own Quick Sync library for certain codecs), but I was keen to find an OpenCV solution for further use with other more complex modules later on.

Then the headaches start

After uploading the HECTOR public beta and the Media modules, I suddenly found that the packaged version of 'Media' wouldn't load into HECTOR. It took quite a while to track it down, but essentially Emgu CV loads a bunch of DLLs when it is first initialized, and it has an internal list of these DLLs. It expects to find them in an 'x86' or 'x64' directory and if it can't find them, it causes an exception. Technically, the Windows subsystem should be capable of finding the DLLs automatically if I installed them into the System32 folder or if I p/invoked AddDllDirectory and put the DLLs in a specific folder, but I found that Emgu CV *still* fires an exception. In the end, I cleared this exception by adding an empty 'x64' folder to my application's folder - Emgu CV seems to be firing an exception if it can't find the folder, not realising that the DLLs can be loaded elsewhere.

The headaches could also be that I do still seem to have the flu

This really highlighted a failing in my HECTOR module approach, in that while I'd successfully managed to cater for DLLs that the HECTOR module calls, I hadn't really catered for the fact that a HECTOR module may call a DLL, which in turn tries to load another external DLL and does it in a way that I hadn't expected. So, what I needed to do was try to cater for these edge cases.

I decided that I didn't like the approach of adding a blank 'x64' folder into my main HECTOR path, it just seems unnecessary and not very tidy. I also felt that Emgu CV's approach of looking for the DLLs only in specific folders and throwing an exception if they're not there is a grey area in terms of best practice - yes, you should throw an error if you can't find the DLLs, but I would prefer it if you didn't throw an error if the DLLs can still be loaded by the Windows subsystem when they are placed in an unexpected location. I believe the loading mechanism could be made more resilient and I'm hoping I can contribute something to the Emgu team when I've given it sufficient thought.

In the meantime, I've forked the Emgu CV code and have added my own method which can be called to manually initialize the Emgu CV library with a different folder for the DLLs. I then created a way of flagging certain files within the module packages as 'external', meaning they need to be unpacked and stored on-disk before the module can run. The running module can then find out where the external files have been unpacked to, so that this information can be used to help Emgu CV (or any other external library) find the files it needs.

I'll need to put my fork of Emgu CV online very soon, but I'm going to wait a couple of days just to make sure I've really thought this through in the best way possible. I think the code I've written into Emgu CV can probably be neatened up a bit more so that Emgu CV's standard method still works and my solution can just sit 'on-top', or even act as an automatic fall-back when the standard method fails.

What about supporting edge cases?

I realise this approach doesn't actually support Emgu CV's way of doing things (because I've had to edit their code), and therefore I'm not really supporting them as an edge case. It's disappointing in some senses that I can't support this scenario, but then again Emgu CV is the one throwing the error even though HECTOR is making sure it can load the DLLs. So, what started out as me trying to support Emgu CV as an edge case has ended with me modifying the Emgu CV code to support me as an edge case instead. I don't think this is a bad thing - what I'm supporting instead is really a way for a programmer to write a module and get third party libraries like Emgu CV working with a minimal amount of code revisions, and also I hope that those code revisions actually improve the third-party library. I think that's as good as I can really get it.

Several new updates

23 August 2017

So, this latest update involves several new modules and some modifications to the code. First up, after uploading the video effects modules in the last update, I wanted a way to preview them out on the web as an animated GIF. I therefore made a quick module that does exactly that - it takes an incoming video stream and outputs it directly to an animated GIF file. Speed is pretty good, it does drop frames in between saving to disk, but I made it fairly resilient in that it'll calculate how many frames are dropped and use the correct amount of delay between each frame.

Of course, this then gave me my next headache - my HD video feeds are way too big to really be of any use as animated GIFs, even if they did work pretty well on my desktop. Well, I'd been meaning to write a video scaler module, so I mocked up a quick scaler with 1x, 0.75x, 0.5x and 0.25x scaling options and then uploaded that as my second brand new HECTOR module in one day.

This of course means I can now scale my HD video feeds down to one quarter of their original size before outputting to the video effects units, and then feed the output from those units through to an animated GIF generator - which gave me the following previews.

VideoEffects.ASCIIArt:

VideoEffects.BadTV:

VideoEffects.BitCrusher:

VideoEffects.BitCrusher, this time with the colours crushed a lot further:

VideoEffects.DotMatrix:

VideoEffects.DoubleVision:

It's worth noting that these were live television feeds, scaled on-the-fly and saved to disk in one operation, with no post-production work required on my part. The GIF dithering could have been done nicer, and I'll be looking into ways of improving that in the future, and of course also getting the frame rate up. But, on the whole, considering I coded up both these modules today in my spare time whilst also recovering from flu, I'm pretty happy with the result.

Did I mention I've got the flu? I've got the flu.

VideoEffects modules and minor update to HECTOR code

18 August 2017

Today I uploaded the first batch of video effects modules for HECTOR, these include a bunch of slightly mad effects like Bad TV and the BitCrusher! alongside some more standard video tools like those used for colour-correction and video cross-fading.

Whilst doing this, I also found and corrected a minor bug which stopped these new modules from loading, and the updated HECTOR v0.11 is now available from the download page. Hopefully after the weekend, I'll get a chance to capture some of the video effects in action and post them up on YouTube.

HECTOR Public Beta Launches!

04 August 2017

After a 7-month development period, HECTOR is finally out of alpha and ready for a public beta. It's still early days, but I'm pleased with the progress so far. HECTOR has a fairly complete web-based UI, multiple HECTOR servers can be networked together, and the base code is capable of extremely fast data processing. I've had it running here with live HD video being run through several effects processes, graphics with dynamic text overlaid on top and then sent out via SDI to an external source, all with very little CPU load.

One of the great things I've been very happy with is the ability to easily add control surfaces to the mix by listening to incoming control sources (such as USB MIDI devices) and assigning the various controllers to controls within HECTOR. It's really easy to set up and opens the door for HECTOR to be used in a variety of live production environments.

Another thing I'm really happy with is the modular system, whereby you download the modules you require for your particular workflow, connect it up however you want and then just let HECTOR run in background from that point onwards. Currently, the available module list is quite small. I've actually made many, many more modules but they have a lot of test code that needs stripping out before I'm ready to put them live. Over the next few weeks, I expect HECTOR to be capable of doing much more than it can right now.

A final point - this week I actually coded up a DMX control module for HECTOR, which shows how adaptable the system can be. Since audio and video can flow through the system as well, this would make it easy to monitor a video or audio signal and run a bank of lighting that automatically takes cues from these signals. More on this soon!

Using HECTOR to monitor video feeds in browser and stream them to other HECTOR machines

31 May 2017

Introduction to HECTOR: Screen capture, USB webcams, Video overlays and MIDI control surfaces

25 May 2017