April 1st CIS565 Progress Summary: Camera and Pathtracing

Here's the first progress summary/blog digest for the GPU Pathtracer project!

Over the past few days, Peter and I established our framework, got random number generation working on the GPU, built an accumulator, figured out parallelized camera ray projection, got spherical intersection tests working, and got a basic path-traced image!

Here are the posts for this update:

1. Random Number Generation: Fun with parallelized random number generators and seeding
2. First Rays on the GPU: Parallel raycasting!
3. Accumulating Iterations: The heart of any monte-carlo based renderer
4. We Have Path Tracing: First working renders!

Here's an image from our very first working render! More soon!

CIS563/CIS565 Final Project Github Repos!

For both MultiFluids and the GPU Pathtracer, we will be making our source code accessibly to all on Github!

Of course commercial coding projects and whatnot have very good reasons for keeping their source code locked down and proprietary, but open source is something I very strongly believe in. Open code allows other people to see what one does and give feedback and suggestions for improvement, and also allows other people interested in similar projects to potentially learn and build off of. Everybody wins!

The MultiFluids repository can be found here: https://github.com/betajippity/MultiFluids

The GPU Pathtracer repository can be found here: https://github.com/peterkutz/GPUPathTracer/

...and of course, the relevant blog posts:

GPU Pathtracer: http://gpupathtracer.blogspot.com/2012/03/github-repository.html

MultiFluids: http://chocolatefudgesyrup.blogspot.com/2012/03/github-and-windowsosx.html

CIS563/CIS565 Final Projects: Multiple Interacting Fluids and GPU Pathtracing

Over the next month and a half, I will be working on a pair of final projects for two of my classes, CIS565 (GPU Programming, taught by Patrick Cozzi), and CIS563 (Physically Based Animation, taught by Joe Kider).

For CIS563, I will be teaming up with my fellow classmate and good friend Dan Knowlton to develop a liquid fluid simulator capable of simulating multiple fluids interacting against each other. Dan is without a doubt one of the best in our class and easily my equal or superior in all things graphics, so working with him should be a lot of fun. Our project is going to be based primarily on the paper Multiple Interacting Fluids by Losasso et. al. and as a starting point we will be using Chris Batty's Fluid 3D framework.

For CIS565, I will be working with my fellow Pixarian and friend Peter Kutz, who is somewhat of a physically based rendering titan at Penn. Working with Peter should be a very interesting and exciting learning experience. Peter and I will be developing a CUDA based GPU Pathtracer with the goal of generating convincing photorealistic images extremely rapidly. We will be developing our GPU pathtracer from scratch, although we will obviously draw inspiration from both Peter's Photorealizer project and my own CPU pathtracer project.

For both projects, we will be keeping blogs where we will post development updates, so I won't post too much about development details to this here personal blog. Instead, I'm thinking about posting a weekly digest of progress on both projects with links to interesting highlights on the project blogs. 

Dan and I will be blogging at http://chocolatefudgesyrup.blogspot.com/. We've titled our project "Chocolate Syrup" for two reasons: firstly, Dan likes to codename his project with types of confectionaries, and secondly, chocolate syrup is one type of highly viscous fluid we aim for our simulator to be able to handle!

Peter and I will be blogging at http://gpupathtracer.blogspot.com/. For now we have decided to call our project "Peter and Karl's GPU Pathtracer", for obvious reasons. 

Details for each project can be found in the first post of each blog, which are the project proposals.

Multiple Interacting Fluids Proposal: http://chocolatefudgesyrup.blogspot.com/2012/03/project-proposal.html

GPU Pathtracer Proposal: http://gpupathtracer.blogspot.com/2012/03/project-proposal.html

Both of these projects should be very very cool, and I'll be posting often to both development blogs!

Pathtracer with KD-Tree

I have finished my KD-Tree rewrite! My new KD-Tree implements the Surface-Area Heuristic for finding optimal splitting planes, and stops splitting once a node has either reached a certain sufficiently small surface area, or has a sufficiently small number of elements contained within itself. Basically, very standard KD-Tree stuff, but this time, properly implemented. As a result, I can now render meshes much quicker than before.

Here's a cow in a Cornell Box. Each iteration of the cow took about 3 minutes, which is a huge improvement over my old raytracer, but still leaves a lot of room for improvement:

 
...and of course, the obligatory Stanford Dragon test. Each iteration took about 4 minutes for both of these images (the second one I let converge for a bit longer than the first one), and I made these renders a bit larger than the cow one:

So! Of course the KD-Tree could still use even more work, but for now it works well enough that I think I'm going to start focusing on other things, such as more interesting BSDFs and other performance enhancements.

First Pathtraced Image!

Behold, the very first image produced using my pathtracer!

Granted, the actual image is not terribly interesting- just a cube inside of a standard Cornell box type setup, but it was rendered entirely using my own pathtracer! Aside from being converted from a BMP file to a PNG, this render has not been modified in any way whatsoever outside of my renderer (I have yet to name it). This render is the result of a thousand iterations. Here are some comparisons of the variance in the render at various iteration levels (click through to the full size versions to get an actual sense of the variance levels):

Upper Left: 1 iteration. Upper Right: 5 iterations. Lower Left: 10 iterations. Lower Right: 15 iterations.

Upper Left: 1 iteration. Upper Right: 250 iterations. Lower Left: 500 iterations. Lower Right: 750 iterations.

Each iteration took about 15 seconds to finish.

Unfortunately, I have not been able to move as quickly with this project as I would like, due to other schoolwork and TAing for CIS277. Nonetheless, here's where I am right now:

Currently the renderer is in a very very basic primitive state. Instead of extending my raytracer, I've opted for a completely from scratch start. The only piece of code brought over from the raytracer was the OBJ mesh system I wrote, since that was written to be fairly modular anyway. Right now my pathtracer works entirely through indirect lighting and only supports diffuse surfaces... like I said, very basic! Adding direct lighting should speed up render convergence, especially for scenes with small light sources. Also, right now the pathtracer only uses single direction pathtracing from the camera into the scene... adding bidirectional pathtracing should lead to another performance boost.

I'm still working on rewriting my KD-tree system, that should be finished within the next few days. 

Something that is fairly high on my list of things to do right now is redesign the architecture for my renderer... right now, for each iteration, the renderer traces a path through a pixel all the way to its recursion depth before moving on to the next pixel. As soon as possible I want to move the renderer to use an iterative (as opposed to recursive) accumulated approach for each iteration (slightly confusing terminology, here i mean iteration as in each render pass), which, oddly enough, is something that my old raytracer already does. I've already started moving towards the accumulated approach; right now, I store the first set of raycasts from the camera and reuse those rays in each iteration.

One cool thing that storing the initial ray cast allows me to do is to generate a z-depth version of the render for "free":

 
Okay, hopefully by my next post I'll have the KD-tree rewrite done!

Smoke Sim: Preconditioning and Huge Grids

I have added preconditioning to my smoke simulator! For the preconditioner, I am using Incomplete Cholesky, which is the preconditioner recommended in chapter 4 of the Bridson Fluid Course Notes. I've also troubleshooted by vorticity implementation, so the simulation should produce more interesting/stable vortices now.

The key reason for implementing the preconditioner is simple: speed. With a faster convergence comes an added bonus: being able to do larger grids due to less time required per solve. Because of that speed increase, I can now run my simulations on 3D grids.

In previous years, the CIS563 smoke simulator framework usually hit a performance cliff at grids beyond around 50x50x50, but last year Peter Kutz managed to push his smoke simulator to 90x90x36 by implementing a sparse A-Matrix structure, as opposed to storing every single data point, including empty ones, for the grid. This year's smoke simulation framework was updated to include some of Peter's improvements, and so Joe reckons that we should be able to push our smoke simulation grids pretty far. I've been scaling up starting from 10x10x10, and now I'm at 100x100x50:

This simulation took about 24 hours to run on a 2008 MacBook Pro with a 2.8 Ghz Core 2 Duo, but that is actually pretty good for fluid simulation! According to my rather un-scientific estimates, the simulation would take about 4 or 5 days without the preconditioner, and even longer without the sparse A-Matrix. I bet I can still push this further, and I'm starting to think about multithreading the simulation with OpenMP to get even more performance and even larger grids. We shall see.

One more thing: rendering this thing. So far I have not been doing any fancy rendering, just using the default OpenGL render that our framework came with. However, I want to get this into my volumetric renderer at some point and maybe even try out the pseudo-black body stuff with it. Eventually I want to try rendering this out with my pathtracer too!

Smoke Simulation Basics!

For CIS563 (Physically Based Animation), our current assignment is to write a fluid simulator capable of simulating smoke inside of a box. For this assignment, we're using a semi-lagrangian approach based on Robert Bridson's 2007 SIGGRAPH Course Notes on Fluid Simulation.

I won't go into the nitty-gritty details of the math behind the simulation (for that, consult the Bridson notes), but I'll give a quick summary. Basically, we start with a specialized grid structure called the MAC (marker and cell) grid, where each grid cell stores information relevant to the point in space the cell represents, such as density, velocity, temperature, etc. We update values across the grid by pretending a particle carried the cell's values into the cell and using the velocity to extrapolate in time the particle's previous position, and look up the values from the grid cell the particle was previously in. We then use that information to perform advection and projection and solve the system through a preconditioned conjugate gradient solver.

So far I have implemented density advection, projection, buoyancy (via temperature advection), and vorticity. For the integration scheme I'm just using basic Eularian, which was the default for the framework we started with. Eularian seems stable enough for the smoke sim, but I might try to go ahead and implement RK4 later anyway, since I suspect RK4 won't smooth out details as much as basic Eularian.

I'm still missing the actual preconditioner, so for now I'm only testing the simulation on a 2D grid, since otherwise the simulation times will be really really long.

Here is a test on a 100x100 2D grid!

Jello Sim Maya Integration

I ported my jello simulation to Maya!

Well, sort of.

Instead of building a full Maya plugin like my good friend Dan Knowlton did, I opted for a simpler approach: I write out the vertex positions for each jello cube for each time step to a giant text file, and then use a custom Python script in Maya to read the vertex positions from the text file and animate a cube inside of Maya. It is a bit hacky and not nearly as elegant as the full-Maya-plugin approach, but it works in a pinch.

I think beng able to integrate my coding projects into artistic projects is very important, since at the end of the day, the main point of computer graphics is to be able to produce a good looking image. As such, I thought putting some jello into my kitchen scene would be fun, so here is the result, rendered out with Vray (some day I want to replace Vray with my own renderer though!):

 

The rendering process I'm using isn't perfect yet... the fact that the jello cubes are being simulated with relatively few vertices is extremely apparent in the above video, as can be seen in how angular the edges of the jello become when it wiggles. At the moment, I can think of two possible fixes: one, simple run the simulation with a higher vertex count, or two, render the jello as a subdivision surface with creased edges. Since the second option should in theory allow for better looking renders without impacting simulation time, I think I will try the subdivision method forst.

But for now, here are some pretty still frames:
 

Multijello Simulation

The first assignment of the semester for CIS563 is to write a jello simulator using a particle-mass-spring system. The basic jello system involves building a particle grid where all of the particles are connected using a variety of springs, such as bend and shear springs, and then applying forces across the spring grid. In order to step the entire simulation forward in time, we also have to implement a stable integration scheme, such as RK4. For each step forward in time, we have to do intersection tests for each particle against solid objects in the simulation, such as the ground plane or boxes or spheres.

The particle-mass-spring we used is based directly on the Baraff/Witkin 2001 SIGGRAPH Physically Based Animation Course Notes.

For the actual assignment, we were only required to support a single jello interacting against boxes, spheres, cylinders, and the ground. However, I think basic primitives are a tad boring... so I went ahead and integrated mesh collisions as well. The mesh collision stuff is actually using the same OBJ mesh system and KD-Tree system that I am using for my pathtracer! I am planning on cleaning up my OBJ/KD-Tree system and releasing it on Github or something soon, as I think I will still find even more uses for it in graphics projects.

Of course, a natural extension of mesh support is jello-on-jello interaction, which is why I call my simulator "multijello" instead of just singular jello. For jello-on-jello, my approach is to update one jello at a time, and for each jello, treat all other jellos in the simulation as just more OBJ meshes. This solution yields pretty good results, although some interpenetration happens if the time step is too large or if jello meshes are too sparse.

Here's a video showcasing some things my jello simulator can do:

Pathtracer Time

This semester I am setting out on an independent study under the direction of Joe Kider to build a pathtracer (obviously inspired by my friend and fellow DMD student Peter Kutz). Global illumination rendering techniques are becoming more and more relevant in industry today as hardware performance in the past few years has begun to reach a point where GI in commercial productions is suddenly no longer unfeasibly expensive. Some houses like Sony Imageworks have already moved to full GI renderers like Arnold, while other studios like Pixar are in the process of adopting GI based renderers or extending their existing renderers to support GI lighting. This industry move, coupled with the fact that GI quite simply produces very pretty results, sparked my initial interesting in GI techniques like pathtracing. Having built a basic raytracer last semester, I decided in typical over-confident style: "how hard could it be?"

Here's my project abstract:

Both path tracing and bidirectional scatter distribution functions (BSDFs) are ideas that have existed within the field of computer graphics for many years and have seen numerous implementations in a variety of rendering pack- ages. Similarly, creating images of convincing plant life is a technical challenge that a host of solutions now exist for. However, achieving dynamic plant effects such as the change of a plants coloring during the transition from summer to fall is a task that to date has been mostly been accomplished using procedural techniques and various compositing tricks.

The goal of this project is to build a path tracing based renderer that is designed specifically with the intent to facil- itate achieving dynamic plant effects with a more physically based approach by introducing a new time component to the existing bidirectional scatter distribution model. By allowing BSDFs to vary over not only space but also over time, plant effects such as leaf decay could be achieved through shaders with appearances that are driven through physically based mathematical models instead of procedural techniques. In other words, this project has two main prongs: develop a robust path tracer with at least basic functionality, and then develop and implement a time- dependent BSDF model within the path tracer.

...and here's some background that I wrote up for my proposal...

1. INTRODUCTION
Efficiently rendering convincing images with direct and indirect lighting has been a major problem in the field of computer graphics since the field’s very inception, as con- vincingly realistic graphics in games and movies depends upon lighting that can accurately mimic that of reality. Known generally as global illumination, the indirect light- ing problem has in the past decade seen a number of solu- tions such as path tracing and photon mapping that can generate convincingly realistic images with reasonable computational resource consumption and efficiency.
One of the key discoveries that enabled the development of modern global illumination techniques is the concept of Bidirectional Scattering Distribution Functions, or BSDFs. Developed as a superset and generalization of two other concepts known as bidirectional reflectance distribution functions (BRDFs) and bidirectional transmittance distribu- tion functions (BTDFs), BSDF is a general mathematical function that describes how light is scattered by a certain surface, given the material properties of the surface. BSDFs are useful today for representing the material properties of an object at a single point in time; however, in reality mate- rial properties can change and morph over time, as exem- plified by the natural phenomena of leaf color changes from summer to fall.
This project will attempt to build a prototype of a path tracing renderer with a BSDF model modified to include an additional time component to allow for material properties to change over time in a way representative of how material properties change over time in reality. The hope is that
such a renderer will prove to be useful in future attempts to recreate natural phenomena using physically based models, such as leaf decay.

...and the actual goal of the project...

1.1 Design Goals
The project’s goal is to develop a reasonably robust and efficient path tracing renderer with a BSDF model modified to include an additional time component. In order to prove the feasibility of such a modified BSDF model, the end goal is to be able to use the renderer to produce images of plant life with changing surface material properties, in addition to standard test image such as Cornell Box tests that validate the functionality of the underlying basic path tracer.

...and finally, what I'm hoping I'll actually be able to produce at the end of this independent study:

1.2 Projects Proposed Features and Functionality
The proposed renderer should allow a user to load a sce- ne with an arbitrary number of lights, materials, and objects and render out a realistic, global illumination based render. The renderer should be able to render implicitly defined objects such as spheres and cubes in addition to meshes defined in the .obj format. The renderer should also allow users to specify changes in object/light/camera transfor- mations over time in addition to changes in materials and BSDFs over time and render out a series of frames showing the scene at various points in time. A graphical interface would be a nice additional feature, but is not a priority of this project.

I'll be posting at least weekly updates to this blog showing my progress. In my next post, I'll go over some of the papers and sources Joe gave me to look over and explain some of the basic mechanics of how a pathtracer works. Apologies for the casual reader for this particular post being extremely text heavy; I shall have images to show soon!

Basic Raytracer and Fun with KD-Trees

The last assignment of the year for CIS460/560 (I'm still not sure what I'm supposed to call that class) is the dreaded RAYTRACER ASSIGNMENT.

The assignment is actually pretty straightforward: implement a recursive, direct lighting only raytracer with support for Blinn-Phong shading and support for basic primitive shapes (spheres, boxes, and polygon extrusions). In other words, pretty much a barebones implementation of the original Turner Whitted raytracing paper.

I've been planning on writing a global-illumination renderer (perhaps based on pathtracing or photon mapping?) for a while now, so my own personal goal with the raytracer project was to use it as a testbed for some things that I know I will need for my GI renderer project. With that in mind, I decided from the start that my raytracer should support rendering OBJ meshes and include some sort of acceleration system for OBJ meshes.

The idea behind the acceleration system goes like this: in the raytracer, one obviously needs to cast rays into the scene and track how they bounce around to get a final image. That means that every ray needs to have intersection tests against objects in the scene in order to determine what ray is hitting what object. Intersection testing against mathematically defined primitives is simple, but OBJ meshes present more of a problem; since an OBJ mesh is composed of a bunch of triangles or polygons, the naive way to intersection test against an OBJ mesh is to check for ray intersections with every single polygon inside of the mesh. This naive approach can get extremely expensive extremely quickly, so a better approach would be to use some sort of spatial data structure to quickly figure out what polygons are within the vicinity of the ray and therefore need intersection testing.

After talking with Joe and trawling around on Wikipedia for a while, I picked a KD-Tree as my spatial data structure for accelerated mesh intersection testing. I won't go into the details of how KD-Trees work, as the Wikipedia article does a better job of it than I ever could. I will note, however, that the main resources I ended up pulling information from while looking up KD-Tree stuff are Wikipedia, Jon McCaffrey's old CIS565 slides on spatial data structure, and the fantastic PBRT book that Joe pointed me towards.

Implementing the KD-Tree for the first time took me the better part of two weeks, mainly because I was misunderstanding how the surface area splitting heuristic works. Unfortunately, I probably can't post actual code for my raytracer, since this is a class assignment that will repeated in future incarnations of the class. However, I can show images!

The KD-Tree meant I could render meshes in a reasonable amount of time, so I rendered an airplane:

The airplane took about a minute or so to render, which got me wondering how well my raytracer would work if I threw the full 500000+ poly Stanford Dragon at it. This render took about five or six minutes to finish (without the KD-Tree in place, this same image takes about 30 minutes to render):

Of course, the natural place to go after one dragon is three dragons. Three dragons took about 15 minutes to render, which is pretty much exactly a three-fold increase over one dragon. That means my renderer's performance scales more or less linearly, which is good.

For fun, and because I like space shuttles, here is a space shuttle. Because the space shuttle has a really low poly count, this image took under a minute to render:

For reflections, I took a slightly different approach from the typical recursive method. The normal recursive approach to a raytracer is to begin with one ray, and trace that ray completely through recursion to its recursion depth limit before moving onto the next pixel and ray. However, such an approach might not actually be idea in a GI renderer. For example, from what I understand, in pathtracing a better raytracing approach is to actually trace everything iteratively; that is, trace the first bounce for all rays and store where the rays are, then trace the second bounce for all rays, then the third, and so on and so forth. Basically, such an approach allows one to set an unlimited trace depth and just let the renderer trace and trace and trace until one stops the renderer, but the corresponding cost of such a system is slightly higher memory usage, since ray positions need to be stored for the previous iteration.

Adding reflections did impact my render times pretty dramatically. I have a suspicion that both my intersection code and my KD-Tree are actually far from ideal, but I'll have to look at that later. Here's a test with reflections with the airplane:

...and here is a test with three reflective dragons. This image took foooorrreeevvveeeerrrr to render.... I actually do not know how long, as I let it run overnight:

I also added support for multiple lights with varying color support:

Here are some more images rendered with my raytracer:

In conclusion, the raytracer was a fun final project. I don't think my raytracer is even remotely suitable for actual production use, and I don't plan on using it for any future projects (unlike my volumetric renderer, which I think I will definitely be using in the future). However, I will definitely be using stuff I learned from the raytracer in my future GI renderer project, such as the KD-tree stuff and the iterative raytracing method. I will probably have to give my KD-tree a total rewrite, since it is really really far from optimal here, so that is something I'll be starting over winter break! Next stop, GI renderer, CIS563, and CIS565!

As an amusing parting note, here is the first proper image I ever got out of my raytracer. Awww yeeeaaahhhhhh:

A Volumetric Renderer for Rendering Volumes

The first assignment of the semester for CIS460 was to write, from scratch in C++, a volumetric renderer. Quite simply, a volumetric renderer is a program that can create a 2D image from a 3D discretized data set. Such data set is more often referred to as a voxel grid. In other words, a volumetric renderer makes pictures from voxels. Such renderers are useful in visualizing medical imaging data and some forms of 3D scans and blah blah blah...

...or you can make pretty clouds.

One of the first things I ever tried to make when I first was introduced to Maya was a cloud. I quickly learned that there simply is no way to get a nice fluffy cloud using polygonal modeling techniques. Ever since then I've kept the idea of making clouds parked in the back of my head, so when we were assigned the task of writing a volumetric renderer that could produce clouds, obviously I was pretty excited. 

The coolest part of studying computer graphics from the computer science side of things has got to be the whole idea of "well, I want to make X, but I can't seem to find any tool that can do X, so I guess.... I'LL JUST WRITE MY OWN PROGRAM TO MAKE X." 

I won't go into detailed specifics about implementing the volumetric renderer, as that is a topic well covered by many papers written by authors much smarter than me. Also, future CIS460 students may stumble across this blog, and half the fun of the assignment is figuring out the detailed implementation for oneself. I don't want to ruin that for them ;) Instead, I'll give a general run-through of how this works. 

The way the volumetric renderer works is pretty simple. You start with a big ol' grid of voxels, called... the voxel grid or voxel buffer. From the camera, you shoot an imaginary ray through each pixel of what will be the final picture and trace that ray to see if it enters the voxel buffer. If the ray does indeed hit the voxel buffer, then you slowly sample along the ray a teeny step at a time and accumulate the color of the pixel based on the densities of the voxels traveled through. Lighting information is easy too: for each voxel reached, figure out how much stuff there is between that voxel and any light sources, and use a fancy equation to weight the amount of shadow a voxel receives. "But where does that voxel grid come from?", you may wonder. In the case of my renderer, the voxel grid can either be loaded in from text files containing voxel data in a custom format, or the grid can be generated by sampling a Perlin noise function for each voxel in the grid. 

So obviously volumetric renderers are pretty good for rendering clouds, as one can simply represent a cloud as a bunch of discrete points where each point has some density value. However, discretizing the world has a distinct disadvantage: artifacting. In the above render, some pixel-y artifacting is visible because the voxel grid I used wasn't sufficiently high resolution enough to make each voxel indistinguishable. The problem is even more obvious in this render, where I stuck the camera right up into a cloud: 

 

(sidenote for those reading out of interest in CIS460: I implemented multiple arbitrary light sources in my renderer, which is where those colors are coming from) 

There are four ways to deal with the artifacting issue. The first is to simply move the camera further away. Once the camera is sufficiently far away, even a relatively low resolution grid will look pretty smooth: 

 

A second way is to simply dramatically increase the resolution of the voxel mesh. This technique can be very very very memory expensive though. Imagine a 100x100x100 voxel grid where each voxel requires 4 bytes of memory... the total memory required is about 3.8 MB, which isn't bad at all. But lets say we want a grid 5 times higher in resolution... a 500^3 grid needs 476 MB! Furthermore, a 1000x1000x1000 grid requires 3.72 GB! Of course, we could try to save memory by only storing non-empty voxels through the use of a hashmap or something, but that is more computationally expensive and gives no benefit in the worst case scenario of every voxel having some density. 

 A third alternative is to use trilinear interpolation or some other interpolation scheme to smooth out the voxel grid as its being sampled. This technique can lead to some fairly nice results: 

At least in the case of my renderer, there is a fourth way to deal with the artifacting: instead of preloading the voxel buffer with values from Perlin noise, why not just get rid of the notion of a discretized voxel buffer altogether and directly sample the Perlin noise function when raymarching? The result would indeed be a perfectly smooth, artifact free render, but the computational cost is extraordinarily high compared to using a voxel buffer.

Of course, one could just box blur the render afterwards as well. But doing so is sort of cheating. 

I also played with trying to get my clouds to self illuminate, with the hope of possibly eventually making explosion type things. Ideally I would have done this by properly implementing a physically accurate black body system, but I did not have much time before the finished assignment was due to implement such a system. So instead, my friend Stewart Hills and I came up with a fake black body system where the emmitance of each voxel was simply determined by how far the voxel is from the outside of the cloud. For each voxel, simply raycast in several random directions until each raymarch hits zero density, pick the shortest distance, and plug that distance into some exponential falloff curve to get the voxel's emittance. Here's a self-glowing cloud: 

 

...not even close to physically accurate, but pretty good looking for a hack that was cooked up in a few hours! A closeup shot: 

So! The volumetric renderer was definitely a fun assignment, and now I've got a cool way to make clouds! Hopefully I'll be able to integrate this renderer into some future projects!

Building/Installing Alembic for OSX

Alembic is a new open-source computer graphics interchange framework being developed by Sony Imageworks and ILM. The basic idea is that moving animation rigs and data and whatnot between packages can be a very tricky procedure since every package has its own way to handle animation, so why not bake out all of that animation data into a common interchange format? So, for example, instead of having to import a Maya rig into Houdini, you could rig/animate in Maya, bake out the animation to Alembic, bring that into Houdini to conduct simulations with, and then bake out the animation and bring it back into Maya. This is a trend that a number of studios including Sony, ILM, Pixar, etc. have been moving toward for some time.

I’ve been working on a project lately (more on that later) that makes use of Alembic, but I found that the only way to actually get Alembic is to build it from source. That’s not terribly difficult, but there’s not really any guides out there for folks who might not be as comfortable with building things from source. So, I wrote up a little guide!

Here’s how to build Alembic for OSX (10.6 and 10.7):

1. Alembic has a lot of dependencies that can be annoying to build/install by hand, so we’re going to cheat and use Homebrew. To install Homebrew:

/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

2. Get/build/install cmake with Homebrew:

brew install cmake

3. Get/build/install Boost with Homebrew:

brew install Boost

4. Get/build/install HDF5 with Homebrew:

brew install HDF5

HDF5 has to make install itself, so this may take some time to run. Be patient.

5. Unfortunately, ilmbase is not a standard UNIX package, so we can’t use Homebrew. We’ll have to build ilmbase manually. Get it from:

http://download.savannah.nongnu.org/releases/openexr/ilmbase-1.0.2.tar.gz

Untar/unzip to a readily accessible directory and cd into the ilmbase directory. Run:

./configure

After that finishes, we get to the annoying part: ilmbase by default makes use of a deprecated GCC 3.x compiler flag called Wno-long-double, which no longer exists in GCC 4.x. We’ll have to deactivate this flag in ilmbase’s makefiles manually in order to build correctly. In each and every of the following files:

/Half/Makefile
/HalfTest/Makefile
/Iex/Makefile
/IexTest/Makefile
/IlmThread/Makefile
/Imath/Makefile
/ImathTest/Makefile

Find the following line:

CXXFLAGS = -g -O2 -D_THREAD_SAFE -Wno-long-double

and delete it from the makefile.

Once all of that is done, you can make and then make install like normal.

Now move the ilmbase folder to somewhere safe. Something like /Developer/Dependencies might work, or alternatively /usr/include/

6. Time to actually build Alembic. Get the source tarball from:

http://code.google.com/p/alembic/wiki/GettingAlembic

Untar/unzip into a readily accessible directory and then create a build root directory parallel to the source root you just created:

mkdir ALEMBIC_BUILD

The build root doesn’t necesarily have to be parallel, but here we’ll assume it is for the sake of consistency.

7. Now cd into ALEMBIC_BUILD and bootstrap the Alembic build process. The bootstrap script is a python script:

python ../[Your Alembic Source Root]/build/bootstrap/alembic_bootstrap.py

The script will ask you for a whole bunch of paths:

For “Please enter the location where you would like to build the Alembic”, enter the full path to your ALEMBIC_BUILD directory.

For “Enter the path to lexical_cast.hpp:”, enter the full path to your lexical_cast.hpp, which should be something like /usr/local/include/boost/lexical_cast.hpp

For “Enter the path to libboost_thread:”, your path should be something like /usr/local/lib/libboost_thread-mt.a

For “Enter the path to zlib.h”, your path should be something like /usr/include/zlib.h

For “Enter the path to libz.a”, we’re actually not going to link against libz.a. We’ll be using libz.dylib instead, which should be at something like /usr/lib/libz.dylib

For “Enter the path to hdf5.h”, your path should be something like /usr/local/include/hdf5.h

For “Enter the path to libhdf5.a”, your path should be something like /usr/local/Cellar/hdf5/1.x.x/lib/libhdf5.a (unless you did not use Homebrew for installing hdf5, in which case libhdf5.a will be in whatever lib directory you installed it to)

For “Enter the path to ImathMath.h”, your path should be something like /usr/local/include/OpenEXR/ImathMath.h

For “Enter the path to libImath.a”, your path should be something like /usr/local/lib/libImath.a

Now hit enter, and let the script finish running!

8. If everything is bootstrapped correctly, you can now make. This will take a while, be patient.

9. Once the make finishes successfully, run make test to check for any problems.

10. Finally, run make install, and we’re done! Alembic should install to something like /usr/bin/alembic-1.x.x/.

Installing Numpy for Maya 2012 64-bit on OSX 10.7

On OSX 10.6, installing Numpy for Maya 2012 was simple enough. You could do it either by directly copying the Numpy install folder into Maya's Python's site-packages folder or by adding a sys.path.append to Maya's UserSetup.py. The process was quite simple since OSX 10.6's default preinstalled version of Python was 2.6.x and Maya 2012 uses Python 2.6.x as well.

However, OSX 10.7 comes with Python 2.7.x, so a few extra steps are needed:

For Maya 2012 64-bit:
1. OSX 10.7 comes with Python 2.7.x, but we need 2.6.x, so install 2.6.x using the official installer from here: http://www.python.org/ftp/python/2.6.6/python-2.6.6-macosx10.3.dmg
2. Since we're using 64-bit Maya with 64-bit Python, we'll need a 64-bit build of Numpy. The official version distributed on scipy.numpy.org is 32-bit, so we'll need a 64-bit build. Thankfully, there is an unofficial 64-bit build in the form of the Scipy Superpack for Mac OSX. Even though we're on OSX 10.7, we'll want the OSX 10.6 variety of the script since the OSX 10.7 is Python 2.7.x dependent: http://idisk.mac.com/fonnesbeck-Public/superpack_10.6_2011.07.10.sh 

EDIT (01/12/2012): I've been informed by Michael Frederickson that the link originally posted to the unofficial 64 bit Scipy Superpack build for 10.6 no longer works. Fortunately, I've backed up both the script and the required dependencies. The install script can be found here: http://yiningkarlli.com/files/osx10.7numpy2.6/superpack_10.6_2011.07.10.sh

3. Go to where the script downloaded to and in Terminal:
chmod +x superpack_10.6_2011.07.10.sh
./superpack_10.6_2011.07.10.sh
If you don't already have GNU Fortran, make sure to answer 'yes' when the script asks.
4. Once the script is done installing, in Terminal:
ls /Library/Python/2.7/site-packages/ | grep numpy
You should get something like: numpy-2.0.0.dev_b5cdaee_20110710-py2.6-macosx-10.6-universal.egg
Even though we installed Numpy for Python 2.6.x, on Lion it installs to the 2.7 folder for some reason. No matter, you can either leave it there or move it to 2.6.
5. Go to /Users/[your username]/Library/Preferences/Autodesk/maya/2012-x64/scripts
6. If you don't have a file named userSetup.py, make one and open it in a text editor. If yes, open it.
7. Add these lines to the file:
import os
import sys
sys.path.append('/Library/Python/2.7/site-packages/[thing you got from step 4]')
8. Sidenote: installing Python 2.6.x sets your default OSX Python to 2.6.x, but if you want to go back to 2.7.x, just edit your ~/.bash_profile and remove these lines:
PATH="/Library/Frameworks/Python.framework/Versions/2.6/bin:${PATH}"
export PATH

....and you should be done! In Maya, you should be able to just use import numpy and you'll be good to go!

Why Backups are Important

Here is a cautionary tale about why backing up one's harddrive is EXTREMELY IMPORTANT.

Over the summer, I started making a little scene based off of the GH House Challenge from RonenBekerman.com, partially as a way to learn Vray and partially just for fun. I was working off of my laptop for the entire project, since I was in California at the time and didn't have access to more powerful machines at home. Being out in California for the summer, I brought as little stuff with me as possible.

One of the things I decided to leave home was my backup Time Machine drive. "Oh, I won't need this over the summer, what are the odds of file corruption or harddrive issues anyhow? I'll be fine", I thought to myself.

Which means, of course, that halfway through the summer a bunch of my files got corrupted and were therefore lost forever, and of course that block of lost data included my in-progress GH House project. NEVER ASSUME THAT YOU DO NOT NEED BACKUP.

What follows are some random in-progress renders that survived through being in posts I made to Facebook and Tumblr.

Here are a series of small in-progress renders showing shading and lighting tests:
 


I also started playing with some ideas for the interior:
 

...and finally, some larger in-progress renders. These renders represent where the project was when I lost all of the data:
 




In the end, the fact that I lost the project isn't as important as the fact that I learned quite a lot from tinkering with this project. However, losing all of the data for this project was definitely a major bummer. But, lesson learned: BACK UP ALL THE TIME.