Tuesday, June 30, 2009

Dimmer Remissions

Tom Dimuzio's side project Dimmer's latest album called Remissions features some cover artwork generated in Studio Artist 4. The cover image was generated using the new version 4 temporal image processing effects.

The process used to generate the cover photo started with taking a video sequence of a moving truck. The video source footage was then processing with the Studio Artist temporal scan tracker to encapsulate the motion in the video over time into a single static image. A diagonal slit scan setting was used for this particular image with a larger frame tracking setting than some of the other examples shown later in this post to give a courser rendition to the motion encapsulation.

Depending on how you setup the temporal scan tracker settings you can generate very different looking static images off of the same video source footage. One example of the same source video footage processed with different temporal scan tracker settings is shown below.

The next image below again uses the same video footage as the previous 2 images but was generated with different temporal scan tracker settings.

Depending on whether the scan tracker's slit scan setting tracks or inversely tracks object motion in the source video footage you can accent and encapsulate object motion in the output image or diminish or remove moving objects from the output image. You can see this in the 2 images above. Note how the motion of the semi-truck is very noticeable in the second image and almost totally removed from the first image.

There are additional temporal image processing effects in Studio Artist that can generate variations on these kinds of motion effects. And the temporal effects can be used to generate static output images or to generate processed video output.

If you would like to check out some of Tom Dimuzio's music or live performances you can check out his web site here.

Vector Output from MSG Effects

One cool new hidden feature in Studio Artist 4 is the ability to generate vector output from some MSG based effects. I forgot to mention this surprise new feature in the previous blog post on the anatomy of a black and white sketch effect. That particular MSG based sketch effect can be output as a raster canvas image or can be output to SVG or PDF vector files for resolution independent display or printing.

This surprise MSG vector output feature is based on some new vector output capabilities built into Studio Artist 4. Previous versions of Studio Artist could output EPS vector files, and that old vector output support is still available. But it only really worked with the vectorizer and the paint synthesizer. And it was only available as a vector output capability as opposed to being based on direct vector drawing to the screen.

Studio Artist 4 has a number of new features that directly incorporate anti-aliased vector drawing. This includes new vector drawing functions in the paint synthesizer. And some new MSG processors that internally incorporate vector drawing into the effects they create. As it turns out, these new vector drawing MSG processors include the 1CAbstract6 and 1CAbstract7 processors used in the black and white sketch preset described in the previous blog post. Another example of a black and white sketch effect derived from MSG processing that can be output as a vector file is shown below.

The new SVG vector output functionality directly ties into the internal vector drawing engine used in Studio Artist 4. So any Studio Artist functions that incorporate internal vector drawing can be directly spooled to a SVG output file while they are being drawn to the screen. The trick is to just record them as steps in a Paint Action Sequence (PASeq). You can then use the Action : Generate SVG : with Paint Action Sequence menu command to generate a SVG vector file from the PASeq preset.

Remember that only action steps that can generate SVG vector output will work when generating SVG vector output. If some action steps do not support vector output then they will draw on the screen but will not generate any output vectors.

The Generate SVG vector output menus also include a Print with Paint Action Sequence menu. The normal File : Print menu command prints from the raster canvas frame buffer. This additional Action :Generate SVG : Print with Paint Action Sequence menu allows you to print directly from the SVG vector generation process when playing a PASeq. There's an option in that standard print dialog that allows for PDF file output (as opposed to printing),so that's how you could generate a PDF file from a MSG preset effect that supports vector output.

Anatomy of a MSG Effect

I recently was asked how the image above was created. It was generated from a source image of a face that was processed in Studio Artist using a 2 step Paint Action Sequence (PASeq). Both action steps in the PASeq are custom MSG presets I constructed that work together to build the overall BW sketch effect.

The PASeq is a factory preset in Studio Artist 4 called Circle Rendition2, so the effect itself is just one of the thousands of preset effects that are built into Studio Artist. But i thought it would be instructive to delve into the construction of this particular effect in detail to give a feel for the hidden power that lurks within Studio Artist.

MSG stands for Modular Synthesized Graphics and is Studio Artist's modular image processing architecture. In version 4 there are over 500 different MSG image processing modules (called processors) that can be configured together to build an infinite variety of different image or video processing effects.

The particular effect discussed in this article is a black and white sketch effect composed of 2 different internal components. They are a thin line sketch that accents facial features and a mass rendering sketch composed of small black circles. These 2 individual stylistic components work together to build up the overall sketch effect. The Circle Rendition2 PASeq preset is shown below, and you can see that it consists of 2 different MSG action steps.

The thin line sketch is actually based on a MSG processor that solves the traveling salesman problem, but instead of generating an optimal travel route between cities it is plotting an optimal sketch route that travels between different facial features. So it's acting like an artist that does a complete sketch by placing the pen on the canvas and then doing a full sketch while never lifting the pen until the sketch is completed. So the entire sketch is composed of only a single continuously drawn path.

The mass sketch component is again based on a different MSG processor, one that tries to represent the mass of an image using individual black circles placed according to an apollonian packing scheme.

The screenshot below of the MSG Advanced Editor shows the processor chain used to construct the first thin line sketch MSG preset.

The TexFV processor is used to generate a texture or edge map from the source image. The Threshold processor then is used to clip the edge map image output of the TexFV processor. By adjusting the Threshold parameter associated with this Threshold processor you can change the complexity or detail of the sketch. The 1CAbstract6 processor is what generates the traveling salesman sketch based on the clipped edge map input image. It's parameter controls are shown below.

The final 1to3 processor just maps the single channel output of the 1CAbstract6 processor to the full RGB color output channels for the MSG preset.

To recap, the green channel of the source input to the MSG preset was processed to generate an edge map, that was then clipped and used as an input to a path generator that connects all of the clipped edge points in an optimal continuous path. The generated path is the thin line sketch effect. An example of the output from this particular MSG preset effect using a different source image is shown below.

Note how the sketch tries to travel to all of the image feature points without lifting the pen. This particular source image also had a lot of edge detail located at the boundaries of the image. You could use additional processing to remove those features from the sketch if desired.

The second MSG preset used to construct the mass sketch based on apollonian packed circles is very similar in nature to the first preset described above. The preset' processor chain can be seen below.

A key difference between this preset and the previous one is that an edge map is not constructed as a part of the effect. Instead the processing takes place directly using the green source channel as input to a ThresholdRamp MSG processor. Again, the previous preset was trying to generate a sketch to represent the source image details (or edges) while this second preset is trying to represent the mass (or luminance) of the source image with black circles. The ThresholdRamp processor acts to boost the contrast of the mass sketch. By adjusting the Threshold and Radius parameters you could control the amount of contrast enhancement, or dial in a more linear tone range to the sketch.

The 1CAbstract7 processor is what generates the apollonian packed circles. Apollonian packing is a mathematical technique for optimally close packing circles into a constrained area, and has a nice stylized appearance. The output of the ThresholdRamp processor is the input to the 1CAbstract7 process that renders it's input using apollonian packed circles. If you didn't care about contrast enhancement you could skip the ThresholdRamp processor step and just use the source image as a direct input into this processor. Again, the last step is a 1to3 processor which just maps the single channel output image into a 3 channel color output for the MSG preset.

The output of this second MSG mass sketch preset is shown below.

Pay close attention to the packing of the circles. the circles touch but never overlap. The final sketch output from the 2 MSG presets discussed above working together to build a finished sketch effect is shown below.

The more technically sophisticated reader may be wondering why the green source channel was used for creating the 2 component sketch effects since it's not really the source luminance but just a crude approximation. You could easily modify these 2 presets to work off of the source image luminance directly. To do this, you would use a RGBtoYIQ processor as the first step in the processor chain, and then use the Y channel output as the input for the subsequent processing steps.

Also, there's no reason why you couldn't construct a single MSG preset that incorporated the internal components of both of the presets described above into a single preset. But the nice thing about using a PASeq to build up an effect like this is that you can work with a library of MSG presets that are easy to understand and build up specific components that work together to create a more complicated effect. Building the PASes is as simple as running the individual processing effects one at a time while PASeq recording is turned on.

Sunday, June 7, 2009

Expanding Vectorizer Region Effects

One powerful feature of the Studio Artist Vectorizer is Region Effects.

There's a wide variety of different stylistic effects you can create by working with different Region Effect Options. Region Effect modify the shapes generated by the vectorizer processing. After stylizing the vector region shapes, when they are overlaid on the canvas they typically will no longer provide total coverage. There are different background color options you can pick to provide a solid color background the stylized shapes can be overlaid on top of. An example of stylized vector shapes on top of a black background is shown below.

Suppose you want to create a stylized vector effect that covers the entire canvas. How can you do this?

There are several different approaches. A simple one is to do 2 vectorizer passes. The first pass uses no region effects. The second pass uses a region effect with the Background Fill option in the vectorizer Drawing control panel set to None. The 2nd pass overlays it's stylized regions on top of the previous non-stylized vectorizer output which does cover the entire canvas.

A different 2-pass approach is to first interpolate the solid background color areas with adjacent region colors using an Image Operation (Ip Op), and then rerun the vectorizer to vectorize the filled in solid color regions. An easy way to do this is to use the Geodesic Interpolation Ip Op. The settings i used to fill in the black background areas by extrapolating the existing color informaiton in the image is shown below.

You want to make sure you use the Pixel Raster Fill setting for the Canvas Render Option if you will be interpolating in the solid background color areas with an Ip Op. You use this setting so that you get hard edges at the region boundaries.

The Vector Anti-Alias setting will provide smooth anti-aliased edges. If you try to use this setting's vectorizer output with the Geodesic Interpolation Ip Op you will fill the solid background areas with the anti-aliased region rim color which is not the same as the solid interior color.

After filling in the solid color background, you may be finished or you may need one more step. If you were just interested in generating a raster version of the vectorizer effect then you are finished. If your goal was to generate vector output like an eps or svg file, then you now need to re vectorize the raster interpolated canvas created by the Geodesic Interpolation Ip Op.

Do do this, you want to use the Flat Color vectorizer Technique, as shown below. This technique assumes the input is already composed of flat colored regions and vectorizes them accordingly. Note that we are using the current layer as the vectorizer source since we are processing the output of the geodesic interpolation ip op that is in the current layer. You also want to make sure you turn off Region Effects when doing this second flat color technique vectorizer pass.

The final stylized vector output is shown below. Note how the stylized image totally covers the canvas with no show through to a solid color background.