Sunday, December 7, 2008

The Joy of Displacement Mapping

Displacement mapping is an extremely useful technique to master and include in your digital effects toolbox. By displacement, we typically mean that individual pixels in the processed image are generated from different spatially modulated positions in the image. So you can think of displacement mapping as being a warp process.

A second image is used to define the spatial modulation (or displacement) of the individual pixels. This means that the modulation image's luminance is used to map to another pixel location. This mapping is the warp taking place for that pixel location in the displacement mapping process. A black modulation pixel would perform no mapping, a white modulation pixel would perform the maximum amount of spatial mapping, gray values in between would perform variable displacement from the min to max values specified for the effect.

There are a number of different ways to generate displacement mapping in Studio Artist. There's the Displacement Image Operation, which can be configured to achieve a wide range of different displacement mapping effects. You can also set the current region selection to the image you wish to modulate with, and then use the Selection Modulate options for Interactive Warp to interactively perform displacement mapping of the current canvas layer by the current selection image. You can choose translate and rotate selection modulate interactive warps.

Displacement mapping an image by itself is a useful effect. Using random noise from the Smart texture Ip Op or the Texture Synthesizer as the modulation image is another useful technique. The Texture Synthesizer has a specific Displace Effect Type that performs displacement mapping based on the texture field being generated by the texture synthesizer. The Displacement Ip Op also has Algorithm options that use uniform random noise or the texture synthesizer output as a modulation source among the many different options provided there.

The Smart Displace Ip Op is another effect that performs displacement mapping, and depending on how you configure it's adjustable parameters can achieve a wide range of different effects. There are also a number of different MSG processors that perform different kinds of displacement mapping.

I recently was asked about how i achieved the effect shown in the image above, which is a single frame from a movie test i posted on the Studio Artist User Forum. The face images in the movie appear to be made out of glass or transparent plastic in front of a colored graffiti static background.

This particular effect was actually achieved by temporal displacement mapping, which means that the face image was used to modulate into a range of different frames of a movie which consisted of a stack of graffiti images. So the displacement for a particular output pixel is being generated by movement through time in a movie as opposed to spatial movement. Temporal displacement is one of the many different Temporal Ip Ops available in Studio Artist 4. However, you can achieve a very similar effect in previous versions of Studio Artist using a more typical spatial displacement mapping in a static graffiti image.

One thing to be aware of to make this particular effect work well is to have the faces or other objects in the source movie you are processing be in front of a flat colored static background. This movie is going to be used as the modulation source for the displacement mapping. You can do this by loading your face movie into the source area. The static image you want to use as the background for the glass effect needs to be loaded into the canvas for each frame of the animation.

In version 3.5 you could use a content keyframe in the layer palette to do this. In version 4 you would use a PASeq Content Context action step. In both cases the same static background image will be loaded into the canvas current layer prior to performing the displacement mapping. The displacement mapping will be performed on the static background image in the current layer, and the source of the displacement modulation will be the source (which in our example is the face movie).

The screen snapshot below shows the face movie loaded into the source area and the editable controls for the Displacement Ip Op.

Note that the Canvas is specified for the To Be Displaced option. The Algorithm is the Fixed Angle, Ip Source Modulates Amt. And the Ip Source is set to the Source Image. So these settings specify that the source movie will displacement modulate the canvas image. The output of these displacement settings applied to a static graffiti background image in the current canvas layer is shown below. The effect created makes the source movie frame appear to be rendered as transparent glass in front of the static graffiti background.

You actually can perform a temporal displacement warp in a movie in Studio Artist 3.5. The trick is to load the movie as a pixel indexed background texture in the paint synthesizer. Pixel Indexed movie background textures index the movie for each pixel of the paint nib being rendered (as opposed to once for the entire nib for a normal movie background texture). So you can set the Paint Fill Setup Fill From option to use the background texture to paint with the modulated background texture movie. With a luminance 1D Frame Mod setting you would be modulating the background texture movie using the source luminance.

You can then use the source luminance to frame index the movie. And you can use region fill as brush pen mode with a Full Screen Generator option in the Region as Brush Mode control panel to fill in the entire canvas with a single paint step. This is a useful trick whenever you want to configure the paint synthesizer to generate effects that process the entire canvas in a single paint step.

We've barely scratched the surface of what you can do with displacement mapping. So it's worth the time to explore this technique and learn all the different things you can do with it. I often use subtle amounts of random noise modulated displacement mapping to subtly distress or age a painted canvas as a final post processing step. The best way to get a handle on this effect is to experiment yourself with different processing options and different source modulators.

Wednesday, November 26, 2008

Iterative Paint Strategies

I've recently been exploring the notion of using Paint Action Sequence (PASeq) keyframing to build iterative paint strategies. As we have discussed in previous posts, a paint strategy is a series of different steps that work together over time to build up a particular artistic style or emulative technique.

Using the PASeq timeline and it's keyframe interpolation capabilities can be a great shortcut to help you build different iterative paint strategies. You can just specify the start and end keyframes for different action steps and then let the PASeq playback parameter interpolation engine do all of the intermediate parameter adjustments for you as the PASeq plays back over a series of frames. As opposed to having to manually make all of those adjustments over the course of the different steps needed to implement your iterative paint strategy. And because the complete paint strategy is encapsulated in a recorded PASeq you can save it as a PASeq preset to use later at the press of a button.

Here's an example of the PASeq i used to construct the paint strategy used to render the image at the top of this post.

The paint strategy iterates over 10 animation frames. The red squares correspond to recorded keyframes. Note that the 'Image Compressor' and the 'PaintBezLayer' action steps have keyframes at the beginning and the end of the animation. This means that the associated parameter values will smoothly interpolate from the start keyframe values to the end keyframe values over the course of the 10 frame animation used to build the iterative paint strategy.

Black keyframes are mute keyframes. When a PASeq action step is muted it will not playback in subsequent frames until another red keyframe is reached as the animation progresses. Note that the mute keyframes in the first 2 action steps are used to setup an initial starting canvas that is then painted over in the various cycles of the iterative paint strategy. The image compressor action step is muted for the first frame only because i want to have dark paint strokes on the canvas before it processes the canvas and i need to have completed one animation cycle to do that.

The concept behind this particular paint strategy is to start with a color simplified canvas and then over paint with bezier paths derived from the source edges. The edge painting starts out highly distorted and over time becomes more and more accurate. The texture synthesizer and 2 water smear and drip paint presets are used to soften and drip the previously painted canvas as a part of each iterative cycle.

You can see the iterative paint strategy progress from start to finish in the animation below.
Note that this particular approach is an example of the more general category of 'loose to tight' paint strategies. This could be as simple as starting with a large brush size to rough in the canvas and then using a progressively smaller brush size over time to fill in increasing detail. This particular example is a little more involved because in addition to a paint step that gets progressively more refined there are also a series of additional steps that modify the previously drawn canvas. This water drip and intelligent texture synth smearing is working to create a complex organic paint texture over the course of the iterative cycles that build up the final output image.

There are endless variations to how you can construct an iterative paint strategy. Another concept i have been exploring recently is to build different tone ranges over the course of the iterative cycle. So you could start at the mid tones and then work over the course of the animation to fill in the highlights and shadows. Or vice versa. Depending on your source image, one direction to build the tone range fill may be more effective than the other. The range of potential effects for this kind of approach can get quite complex when you introduce additional action steps that modify the previously drawn canvas as a part of the iterative cycle.

Tuesday, November 18, 2008

Brick Wall Photo Mosaic - Example Presets

Movie Brushes are a powerful feature in the Studio Artist paint synthesizer. You can use them to create endless varieties of textural patterns in your paintings and photo mosaic imagery created with the paint synthesizer. This could take the form of full photo mosaic effects that try to reproduce the source image with a series of sub images that are appropriately positioned to represent the source image. Or it could be used for more subtle background texturing where the individual movie images are colorized on the fly by the paint synthesizer.

In response to some questions in a Studio Artist User Forum post recently about building brick wall textures i decided to post some preset examples. These examples are all Studio Artist 3.5 presets and you can download them here.

All of these examples use the same brickBrush movie file for a movie source brush. The brickBrush movie was created by first placing a series of photos of brick walls in a folder. I then made a simple Paint Action Sequence (PASeq) that sets the canvas to the source image. I then ran an Action menu command to process a folder of images with the PASeq to a movie file. This generates a movie file that contains all of the images in the folder you choose to process, in this case the brick wall images i wanted to use for my movie brush. You can run the File : Paint Synthesizer : New Movie Brush menu to open a movie file as a new movie source brush.

The different presets show off different mosaic styles you can create using the region as brush pen mode in conjunction with Studio Artist's path start regionization features. Path Start regionization has been discussed on this blog recently. When running a path start regionization preset, Studio Artist first visually analyses your source image and then paints it as a series of individual regions. The painting could be done with paint strokes that fill the generated regions in various ways, or in the case of these preset examples the regions themselves are filled in with individual movie frames from the movie source brush.

The trick with using a movie source brush with the region as brush pen modes is to set the Brush Option control in the Region as Brush Mode control panel to Alpha Only. Normally the Region as Brush pen mode replaces the source brush with it's generated region when drawing. But you don't want to replace your movie source brush, just build an alpha channel for it, and this setting will do that. Of course, you need to switch your Brush Type to Source Brush Alpha to take advantage of this. And you always need to make sure the PathStartRegionize control in the Region as Brush Mode control panel is turned on when using path start regionization with this pen mode.

In addition to the different paint presets included in the preset example download, there is also the movie file used for the movie source brush (called brickBrush). You should place the brickBrush file in the Brush folder in your main Studio Artist folder so that Studio Artist will be able to find it when running these presets. If you don't, you will get a dialog asking you to find the movie file.

There is also a paseq preset that shows how to build regionization painting off of regions generated by the vectorizer. This technique on working with the vectorizer to define regions for the paint synthesizer was discussed in this previous blog entry.

By using the run the File : Paint Synthesizer : New Movie Brush menu you can load your own custom movie files into any of these presets and build photo mosaic effects with your own set of custom images. There are many different ways to build photo mosaic effects in the Studio Artist paint synthesizer. Check out the Graffiti Brushes category in the 3.5 Collection of factory paint synthesizer presets for a completely different example of how to work with movie brushes to create mosaic effects.

Monday, November 17, 2008

Constructing a Paint Strategy

I had a conversation with a Studio Artist user recently about different approaches to building 'paint strategies', and thought it was worth repeating some of that conversation here for other users. By 'paint strategies', i mean the different approaches one can take to building up a custom art process to create some kind of finished piece of art in a particular aesthetic style. Often paint strategies are about trying to emulate traditional art techniques, but there's nothing that restricts you to emulating existing techniques. The goal is simply to figure out how to get from point A (a blank canvas) to point B (the finished painting in the particular style you want to achieve).

A common technique one would use if working with real paint is to first rough in the image with a large brush, and then add detail with progressively smaller brushes. Building this kind of paint strategy in a Paint Action Sequence (PASeq) is pretty straightforward. You start with the preset of your choice using a large brush size, and record an initial PASeq action step that roughs in the background of your painting. Then you progressively edit the max brush size in the Brush Modulation control panel to reduce the brush size and record additional PASeq action steps to build detail.

It's often common to reduce the path length (path shape control panel) as well as the brush size (brush modulation control panel) when building detail. You could also work with Path Start and Path End parameters to focus the painting on the edge structure of the source image. There are some paint synthesizer macro edits to help you do this if you are unfamiliar with paint synth editing. There was a tutorial presented here recently on abstraction vs realism in painting, and any of the detail focusing techniques discussed in that tutorial could also be used as a part of your paint strategy to build detail in your finished painting.

If you are working with manual painting as opposed to action painting, you could edit an autodraw interactive paint preset so that pen pressure or tilt modulates the max brush size and path length. Then you can directly control the resolution of the autodrawn paint strokes as you work with the pen over time to start out roughing in the image and then to build detail. The Digital Street Artist video tutorials show off this approach.

One alternative to roughing in an image is to start with a lower resolution canvas, paint it in, use Supersizer interpolation to increase (upsize) the canvas resolution, and then paint in detail in the higher resolution canvas with your original paint preset. For some kinds of paint effects, using an image operation like Color Simplify might be an alternative to using a paint preset for an initial roughing in of the canvas that then gets painted on top of with subsequent strategy steps.

Another approach to roughing in is to use path start regionization. Path Start Regionization works by analyzing the source image, breaking it up into different regions, and then painting them all in a single paint action step. Depending on what you are trying to achieve, this might be a better approach to roughing in the image than the normal scattered paint strategy of most paint presets. There was a tutorial posted recently that discusses some paint regionization techniques.

One example of emulating a traditional art process might be reproducing a custom watercolor paint effect. In the real world, you might start with a textured paper and apply a colored background wash with a large brush. Then you might fill in details with a finer brush. Detailed edges might then be drawn with a fine black pen or pencil line. This series of different painting steps could be thought of as a watercolor paint strategy.

In the example above, the textured water color paper could be simulated by designing a background texture in the paint synthesizer. There are a number of ways to do this.

You could adjust the settings on the Background Texture parameter pane to simulate a particular paper texture. Or, try scanning a real paper texture and load that into the Background Texture as an image texture. As a different approach, build a Paint Action that applies a random brush to the canvas with a random scan pattern. You could even build a series of image processing steps that generate a textured Canvas. This Canvas could then be loaded into the current region, and used to modulate the Paint Fill.

There are a large number of factory presets that could be used to generate a blurry watercolor background wash. The brush size could then be adjusted to provide a smaller, harder edge brush. The Path Start and Path End parameters could be set so that autodrawn paths would follow the source image edges. Path length and brush size could be reduced again for more detail rendition. The paint source color could be switched to black for the fine detail edge black lines. Then, a Canvas Spread Water Preset could be chosen for a final water wash.

All of these individual editing decisions and subsequent paint actions could be recorded as individual paint action steps in a paint action sequence. The nice thing about recording your custom paint strategy in a PASeq is that you can save it as a preset and then use it at a later time when you want to reproduce your custom style using a new source image. Or you can auto-rotoscope a movie file using the PASeq to generate an output movie rendered with your custom technique.

In addition to developing sets of paint strategies for generating different art styles, you might also try building finishing or optimization strategies. For example, after i finish a painting or effected image there are usually a series of optimization steps i take to clean up and enhance the final image prior to saving it. Depending on the visual characteristics of the painted canvas i might use different finishing approaches. Again, the way to construct an optimization or finishing strategy is to figure out how to encapsulate a series of processing steps to get from point A to point B. For me this usually involves constructing approaches to optimizing contrast and edge rendition in the final image while possibly reducing noise or edge artifacts. I want to do this without enhancing noise or introducing noticeable artifacts into the final image. Sometimes i might add subtle lighting, texturing, or weathering effects.

I used to run my finishing strategies manually by making live edits in the various operation editors and then pressing action. But recently i've become a big fan of working with prebuilt finishing presets. Building a set of PASeqs that encapsulate your different finishing approaches and then accessing them as preset favorites can be a great time saver when working. It also provides an easy way to quickly try different potential finishing approaches on a canvas and undo any choices you are aren't happy with.

Wednesday, November 12, 2008

Abstract Generative Art Strategies

I've recently been experimenting with procedural strategies for creating generative abstract art. There are a lot of different approaches you can take within Studio Artist to create different styles of generative art. For example, there have been a number of different posts here on directed evolution of MSG processors. But you can also use the paint synthesizer to create generative art. Either by itself or in combination with other Studio Artist operation modes.

The image below is an example of how you can use the paint synthesizer to create a generative pattern.

Here's a link to a forum blog entry that discusses how i programmed the paint synthesizer to create this kind of abstract generative image with 4 different paint steps in a PASeq. The coloring for this particular image was created with a static color chosen in the fixed color picker. The slight variations in coloring are a function of the randomization controls in paint color source and modulation control panels.

A different approach is to use a source color palette. The image below is a similar kind of generative system created with 2 paint steps.

The first places a small random mark. The second is a generative preset that grows circular patterns off of the initial random mark. I used the blanking buffer as opposed to white space like in the blog link i mentioned above to define the path start and path end conditions for the generative drawing. I did this so that i could run the generative process multiple times to build up a over-painted canvas. Each paint stroke is painted using the random source palette option, which randomly selects a palette color to paint with.

The example below was created by repeated applications of a generative paint preset, followed by different ip op or texture synthesizer options that feather the painted canvas.

The coloring is derived off of a single source color palette. Note that while this allows for a more complex coloring style than the use of a single fixed source color, there is a spatial uniformity associated with using a single color palette across the entire painted canvas.

An alternative strategy is to use different color palettes in different spatial areas of the canvas. To do this i used a new paint color source control panel option in version 4 that auto-generates the source color palette based on the current paint color. So i turned on this new feature in the paint step that defines the initial mark for the generative pattern to grow off of. And i used a source image for this particular step. Each time this step is run a single mark is placed based on the source image coloring and then the source color gradient is randomly generated off this single image color. The second generative paint step then used the auto-generated color palette to draw it's generative pattern.

The examples below and at the top of the page shows an abstract generative image generated with this coloring technique. Note that the end result for the canvas coloring is more richly varied as opposed to using a single color palette for the entire image.

In version 3.5 of Studio Artist you could construct a PASeq that encapsulates a complete cycle of the iterative generative design process described above. And then run a PASeq animation over multiple frames to build up the final painted image. Since i was using a version 4 beta i used a new feature called Gallery Show, which allows for setting up self running gallery show based on custom collections of presets. The movie example below shows the process involved in building up the final painting as the self running gallery show progresses.


There are an infinite variety of different ways to build generative systems in Studio Artist. One simple variation on the above approach would be to spatially modulate the generative paint patterning in addition to the coloring.

Tuesday, October 28, 2008

Building Hatching Patterns from Embedded Bezier Paths

The previous post talked about how to build hatching patterns from Movie Pixel Indexing Background Textures in the Paint Synthesizer. But there are many other approaches to generating hatching patterns in the paint synthesizer. This post will explore the use of embedded bezier paths to define custom hatching patterns.

The particular factory preset we will be examining is the BezSmartHatch1 and is located in the Default : Auto Sketch category. It is designed to draw on a white canvas, so make sure you erase the canvas to white before using it. The example image at the top shows off how this paint preset renders an image in black and white. The hatching patterning is defined by a set of bezier paths stored in the internal path frame memory of the paint synthesizer. The set of 4 bezier paths used in this preset are shown below.

The bezier paths are used to define the path shape the paint synthesizer draws. If you look in the Path Shape control panel you will see that the Path Type is set to Internal Path Memory 1. This path type draws the complete set of bezier paths in the internal path memory for each auto-generated paint stroke. Using the Random or Cycle Internal Path Memory options would randomly draw 1 of the set of bezier paths, or cycle through them sequentially as individual auto-generated paths are drawn.

Blind drawing sets of hatch marks would eventually fill in the entire canvas with black hatching, so there needs to be an intelligent mechanism turned on to determine where to draw individual paths or more importantly when to stop drawing individual paths to best represent the source image. There are a lot of different ways to program this kind of behavior in the paint synthesizer. One could intelligently work with path start and path end settings to achieve this goal. However, this particular preset uses a WhiteCanvasErr Probability setting in the Path Application control panel.

When this particular setting is activated, before an individual brush nib is rendered onto the canvas, Studio Artist determines if adding it will properly represent the source image at that canvas location. If this is true, then the nib is drawn. If Studio Artist thinks the nib will make the canvas too dark at that location, it skips drawing that particular nib and moves onto the next one. Remember, the settings in the Path Application control panel determining how nibs are applied from the generated internal path to the canvas.

To modify this paint preset to use your own custom bezier hatching, just go to Bezier Draw operation mode. Then draw a set of bezier paths you want to define your custom hatch pattern. Then record the current bezier path frame into the internal bezier path memory. If you now run the preset dsscussed above it will render the source image using your own unique custom bezier hatching pattern.

To illustrate this i drew the following custom hatch pattern in Bezier draw operation mode.

After recording this into the internal bezier path memory, i then ran the editied paint preset and got the following result. Note that the hatching has changed from the square patterning used in the original preset to the circles i used to define my custom bezier hatch pattern.

Version 4 has a new Path Shape feature associated with embedded bezier paths. This is the Cycle Internal Path Memory1 path type option. When this option is used to generate path shapes in the paint synthesizer, the set of individual bezier paths stored in the internal path memory are used as keyframes in an interpolated animation as individual path shapes are drawn. So you can define a set of keyframes for an animation and then have the paint synthesizer generate a complete interpolated animation off of those individual keyframes as it draws auto-generated paths. This can be quite effective when used with Time Particles.

Saturday, October 25, 2008

Custom Hatching Using Pixel Indexed Movie Background Textures

Many people aren't aware of a powerful feature associated with the Background Texture control panel in the Studio Artist paint synthesizer. One of the background texture types is called 'Movie Pixel Index'. It's one of 2 different background texture types that work with loaded movie files. What's unique about the Movie Pixel Index texture type is that the movie frame indexing for the texture movie file is updated for each pixel as the paint synthesizer renders a brush nib. This allows for pixel accurate hatching patterns to be derived off of the individual movie frames.

The image at the top of the page is an example of how you can use movie pixel indexing to build a paint preset that generates a black and white hatching effect. This preset called Circle Hatcher1 is included in your factory paint preset in the Default : general category. It uses a movie brush for the background texture called tex_circle1 that is stored in your Brush folder.

If you look at the individual frames of that movie file you will see that each frame corresponds to a hatching pattern built up from a small number of hatching circle representing a very light gray tone value to a large number of hatching circles representing a very dark tone value. By building your own movie file that defines your own desired set of hatching patterns you can easily modify this default preset to draw with your own unique hatching patterns. One frame of the hatch pattern movie is shown below.

In order to make the hatching patterns seamlessly tile, you can turn on the Path End popup control called Wrap Path at Edge.

Turning on this control will insure that any paint paths you draw wrap at the canvas edges to finish on the other side of the path, leading to a seamless tile.

The hatch patterns in the Circle Hatcher1 preset discussed above were generated using auto-painting with the paint synthesizer circle path shape and a 1 pixel black pen preset. But you are free to hand draw your own hatching patterns if you wish, and you can use any paint presets you want to build up the hatching patterns.

To build you own custom hatching movie file, start by setting the canvas size to something reasonable for a background texture movie brush. For this example i'll set the canvas size to 64x64 pixel. Then open a new movie stream. Now draw your hatching marks one at a time. You can use the cmnd F shortcut to write frames out to the open movie stream as you build up your hatching pattern. Each frame you write out will be used to represent a constant gray level when you render an image with your custom hatching movie background texture. I used a pencil paint preset and hand drew rough x's to build up a more organic hatching pattern, as shown below for one frame.

Each time i drew a new x hatch i wrote out a new movie frame. After i had filled in my small hatch pattern canvas with x's i closed the open movie stream.

I then imported the Circle Hatcher1 paint preset and then ran the File : Paint Synthesizer : New Background Movie Texture menu to replace the movie file used in that factory preset with my own. I selected the custom hatching movie file i just generated. I then went to the Background Texture control panel in the paint synthesizer and changed the Texture Type popup from Movie to Movie Pixel Index. When i ran the edited paint preset i generated the image below that is based on my own custom hatching patterns i rendered out to my hatching movie file.

You want to try to position your individual hatching patterns as evenly as possible when building up the individual movie frames to avoid introducing repetitive patterning into the final rendered image. You can also experiment with different movie frame sizes for the background texture.

Note that you want to use the Fixed Orient Mod and Global Tracking settings for your Movie Pixel Index background texture. These tracking settings insure that the texture tiles as opposed to some other option like tracking the pen orientation.

This example deals with black and white hatching patterns. However, you could build up color hatching patterns if you wish. Duo-tones or tri-tone patterning can be particularly effective to reproduce simulation of chromolith printing.

Tuesday, October 21, 2008

Abstraction vs Realism in Paintings

I question i often get asked when demoing Studio Artist is something along the lines of, 'I love that paint effect, but is there any way to make it more representational of the original source image?' An easy solution to this question is to just mix some of the original source image into the painting. But there are also many different editing techniques you can use to modify the way paint presets draw to make them more representational.

The example shown at the top is using a technique called Local Color Range masking. You can turn this on in the Paint Fill Apply control panel by setting the Nib Masking control to 'Local Color Range'. The image below was made with the same paint preset as the image at top. The only different is that the top image was made with Local Color Range Nib Masking turned on and the bottom image was made with Nib Masking set to Normal (which mean there is no nib masking going on while painting).

Note that the top example does a much better job of reproducing edge detail associated with the original source image.

You can also use local color range masking as a Path End parameter. Doing so will mask your paint paths, leadings to a tighter painting without the artificial nib masking seen when you mask the actual paint nib itself. Which technique is appropriate depends on what you are trying to achieve in your final image.

There are lots of additional paint synthesizer editing techniques you can use to tighten up a painting. You could work with path start and path end settings to restrict painting to specific source luminance or texture ranges. You can work with progressively smaller brush sizes and shorter path lengths over time to build detail in a painting. Like a real artist would do when they start with a large brush to rough in a painting and then use progressively smaller brush sizes to fill in more detail information.

Selection masking is also a useful approach to helping define edge structure in a painting. The image below was created by using the Smart Contrast Ip Op to generate a selection mask, and then path masking a soft airbrush effect.

Artists also combine different media types that work together to create finished images. So, you could start off with an abstract rendition of an image to define mass and then use something like an edge sketch superimposed on top of the rough mass painting to define the edges and make the abstract mass image more representational. The example below shows the addition of an edge sketch to the initial abstract painting (no local color masking) shown in the second image from the top.

The Vectorizer outline technique can also be used to superimpose more defined edge structure on top of an abstract painting. The image below was created by first using a very abstract paint effect, followed by 2 different applications of a vectorizer outline.

All of the techniques described above are really concerned with different approaches to creating tighter edge reproduction in a painting. This is really a key component of how realistic a viewer will perceive a representation painting. So even if you are starting with a paint preset that generates a very abstract rendition of the source image, there are a wide range of different techniques like the ones we discussed above you can use to modify or complement your abstract paint preset to generate more realistic paintings.

Sunday, October 19, 2008

Paint Synth Regionization

There are a number of different path start generators in the Studio Artist Paint Synthesizer that support paint regionization. What this means is that the paint synthesizer analyzes your source image and then breaks it into a series of different geometric regions and paints in each region individually. This is an alternative approach to the individual 1 paint stroke at a time auto-rotoscoping capabilities of the paint synthesizer that you typically use when you press the Action button while in the paint synthesizer.

Many people are unaware that you can extend the stylistic capabilities of paint synth regionization by incorporating the Studio Artist Vectorizer into the paint regionization process. The trick to using the Vectorizer to define the regions you wish to automatically paint is to route the output of the Vectorizer into the current region selection. And to then use the 'selection as regions- all levels' path start generator. In version 4 you can just set the vectorizer composite option to replace the region selection. In 3.5 you would need to run the vectorizer and then set the selection to the vectorized canvas.

When working with paint synth regionization i typically use one of the paint synthesizer macros associated with regionization (like 'auto regionize loose') to quickly reconfigure a given paint preset to draw as a regionization process. I can then make any individual edits needed to this macro edited preset, like setting the path start generator to 'selection as regions- all levels' for this particular example.

The images at the top and bottom of this post show off using the vectorizer to define the regions used for paint synth regionization. The top example was using the vectorizer 'simplify extreme' region effect and the bottom examples was using the 'straighten extreme' region effect. Since the vectorizer has many different editable controls that can affect it's stylistic output, using it in conjunction with paint synth regionization greatly increases the range of possible effects you can generate when auto-painting this way.

Tuesday, October 14, 2008

Studio Artist at Honolulu Mactoberfest

Studio Artist creator John Dalton will be giving a 1 hour presentation on Digital Creativity and Studio Artist at HMAUS's Mactoberfest on Saturday, October 18th from 11-12 am. Mactoberfest is a free and public event celebrating mac computer culture. Everyone interested in Studio Artist or Apple computers is urged to attend. For more information on Mactoberfest and HMAUS you can go here.

Mactoberfest is a part of Hawaii Geek Week, which has it's own web site here.

Monday, October 13, 2008

Canadian Premiere of Mugenkei

October 14th is the Canadian premiere of the Studio Artist animation "Mugenkei" at the Festival Nouveau Cinéma in Montréal. The film is by Studio Artist user Jean Detheux and features the music of German composer Wilfried Jentzsch. The film shows Oct 14th and Oct 18th. You can learn more about the festival here, and info on the "Focus 1 - Mémoires" program including Jean's film is here.

Jean has another Studio Artist film entitled "Shade Lost" which will be the official opener of the New Music Festival Thursday, Oct 23rd in Bowling Green, Ohio. The film will be shown mute while the Eastman Triana will perform the music live as the film is presented. You can learn more about the festival here.

Tuesday, October 7, 2008

MSG Path Start Generators

We've recently been profiling some different ways to utilize MSG presets within the Studio Artist Paint Synthesizer. Another approach to using MSG's modular image processing architecture to extend the functionality of the paint synthesizer is the MSG Scan Generator option in the Path Start control panel.

Some MSG processors that generate a series of internal scan coordinates as a part of their internal processing can make the scan generation available to external programs. All of the chaotic attractor MSG processors like the IFS generators support this feature. So you can use the chaotic attractor to generate a series of path start coordinates within the paint synthesizer when using this option. The first processor in the MSG processor chain needs to be a processor that supports this feature in order for this to work.

The screen snap below of the MSG advanced editor processor chain list shows a simple chaotic attractor preset that uses the IFSGen2 processor, followed by a 1to3 processor to convert the BW single channel attractor image into a 3 channel rgb output. The MSG preset was used as a MSG path start generator for this tutorial.

The example image at the top of the page is a screen snapshot from a paint animation. Directed evolution was used to create a series of different ISF chaotic attractors. A paint preset was built that used the MSG Scan path start generator. Paint Action keyframes were recorded in a Paint Action Sequence within Studio Artist to build a keyframed timeline for the desired animation. This was then rendered out using the Action : Animate with Paint Action Sequence : to Movie menu command.

The end result is an animating chaotic attractor that is painted by the paint synthesizer over the course of the the animation. By overpainting on top of a ip op modified previous canvas image a softer more organic effect was created for the generated paint animation.

Saturday, October 4, 2008

MSG Time Particles

The example above is a frame from an animation created using time particles in the Studio Artist paint synthesizer. Each time particle is an instantiation of a MSG preset. The particular preset used in this example is the same IFS chaotic attractor preset used in the previous post on MSG Live Source Brushes. This means that each time particle is a dynamic IFS fractal animating with movement over time.

Time particles give temporal continuity to individual paint strokes in an animation. The path start control settings are used to specify the individual time particle positions for the first frame in the animation. But for subsequent frames the existing set of time particles move spatially based on the adjustable parameters in the 2 time particle control panels.

Time particles are the mechanism you can use to animate multiple instances of a MSG preset in an animation. In this particular animation we used a MSG Live Source Brush. The alternate approach would be to use a MSG Brush Load. Both options allow you to embed a MSG preset within a paint synthesizer preset.

The Paint Action Sequence (PASeq) used to generate the animation is shown below.

The next to last AutoPaint step is the paint synthesizer step that displays the time particles. The other steps in the PASeq were designed to fade and dissipate previous frame images to give a drifting smoke effect with some gravitation pull. The Geodesic Displacement ip op was used to simulate the gravitation pull, and the Geodesic Warp ip op was used to create the dissipation effect. By modifying the previously drawn frame and then overdrawing the time particles on top of it we introduce temporal continuity into the animation, reducing flicker that could occur if we erased to black each frame.

The name of the last PASeq action step is a little misleading. We used the Blur ip op with an Edge 1 composite setting to create an enhancement effect. This boosts the contrast of the rendered frame prior to it being output to a movie file.

By using multiple paint action steps in a PASeq that have different MSG presets embedded in them you could create animations using different kinds of MSG effects working together to create an overall aesthetic.

Wednesday, October 1, 2008

MSG Live Source Brush

The image above was created using a MSG preset that generates an IFS chaotic attractor as a MSG Live Source Brush. What this means is that the MSG preset is used to generate the source brush dynamically as you paint. This is a new feature in Studio Artist 4.

Note that this is different than the previous post on painting with pickover chaotic attractors. In those examples we showed how to use a MSG preset to create a movie brush with embedded alpha and then paint with that movie brush. So in that example the chaotic attractor images created from the MSG preset are initially generated and then stored in a movie brush. In this example the MSG preset generates the paint synthesizer source brush dynamically on the fly as you paint.

An advantage of the live approach is that you can interactively modulate different MSG processor parameters using things like pen pressure, tilt, orientation, etc. You can also use temporal generators (TG) to dynamically modulate MSG processor parameters. So you can generate a dynamic paint brush based on the MSG preset that can either modulate under your interactive control or proceduraly vary over time or both. So the brush shape can always be unique and different with each paint stroke.

The IFS chaotic attractor was generated using the simple MSG preset shown below.

Some of the IFSGen2's adjustable parameters were modulated by the pen pressure and tilt orientation, others were modulated with procedural temporal generators. Because of this modulation each pen nib is dynamically generated from the MSG preset and has a unique shape as seen below.

The Source Brush control panel was configured to use the new MSG Live source brush.

The MSG TG Option parameters is set to path-random so that the temporal generators in the MSG preset modulate as the path advances (as opposed to modulating with increasing frame time). This and the interactive pen modulation is what drives the unique shapes generated for each paint nib.

Of course there's a lot more you can do with MSG Live Source Brushes besides painting with chaotic attractors. There's really unlimited potential for creating custom paint brush effects depending on how you configure the over 500 MSG processors in your MSG live source brush.

Sunday, September 28, 2008

Painting With Pickover Chaotic Attractors

I've been doing some experiments recently building paint presets that paint with actual chaotic attractors, as seen in the image above. This example showcases how you can use different Studio Artist features working together to build an overall artistic process.

I started out by building a MSG preset that renders a pickover chaotic attractor, as seen in the 2 example images below.

Chaotic attractors are a kind of fractal image, and i really liked the frenetic quality of the attractor and thought it would make a good paint brush due to its splattery appearance.

All of the different chaotic attractor images were created using the same MSG preset shown below.

By adjusting the different editable parameters associated with the MSG preset you can create an infinite number of different attractor images. I used directed evolution via parameter mutation in the Evolution editor in Studio Artist 4 to create a series of different attractor images.

The first MSG processor creates the actual pickover chaotic attractor as shown below. Note that the attractor itself is a black and white image.

The second and third MSG processors are working to create an alpha mask for the attractor as shown below. The black part of the preview image is the mask.

Each attractor image was saved as an individual frame in a movie file. The alpha mask for each attractor was used to create an embedded alpha channel in my movie brush. What the embedded alpha channel and associated alpha mask does is create a matte for each attractor. So when i build a movie brush in the paint synthesizer from my attractor movie i can get individual matted shapes based on the shape of the attractor, as shown below.

The last 2 processors in this MSG preset blur the original black and white attractor and then map that blurred image through a color gradient.

These last 2 steps work to provide the color and give the attractor more dimensionality.

Once i had my attractor movie i then loaded it into the paint synthesizer as a movie source brush. I used a source alpha brush type so that the alpha masks associated with each attractor movie frame were used to generate the brush shape. I used a image processing brush load to create the paint nib, as shown below.

This takes the individual movie source brush frames as the source for the brush load and colorizes then via the paint color mean algorithm.

I used a path shape length of 1 for this movie brush paint preset so 1 paint nib is drawn for each paint stroke. Each paint nib consists of a single chaotic attractor stored as a frame in my movie source brush. The alpha mask associated with each attractor movie frame defines the shape of the paint nib. The individual brush nibs defined by the attractor movie frames and their associated alpha masks can be seen below.

This image below is another example created with this chaotic attractor paint preset.

I ran the preset in 2 different passes. The first roughs in the image. For the second pass i turned on local image range nib masking in the paint fill apply control panel to get tight edge rendition.

I think this example showcases the extreme power of Studio Artist as a creative environment to create digital art. Each operation mode of Studio Artist is very powerful, but when you start to combine them together you can really unleash your imagination to create unique and custom art processes. By using different source image and different msg presets i could revisit the 'process' described above to create an infinite variety of different artistic imagery.