Photogrammetry, as its name implies, is a technique that uses a set of images to obtain 3D model data. By taking photographs from at least two different locations, ‘lines of sight’, sometimes known as rays due to their optical nature, are developed from each camera to point to the object. Their mathematical intersections when this occurs are what produce the three-dimensional coordinates of the points of interest. Thus, the fundamental medium for metrology here is photography. Over the years, the uses of photogrammetry have expanded greatly and have played a critical role in our understanding of faraway objects and the Earth’s surface. It has also led to the modification and improvement of many powerful technologies within construction, engineering, medicine and more. In this lesson, we focused on why it is an innovative, emerging technique within VFX and how it is useful for capturing hero & background assets and reference geometry.
The Process
Three main phases make up the process of capturing and utilising the 3D model. Firstly, there is capture, which involves photographic and recording the object to create the data set. Afterwards, processing follows, when converting and cleaning up the data into a finalised series of images is required. Finally, reconstruction consists of converting the 2D data to 3D through alignment, extraction and reformation. I’ll be going into more depth with these later but we were shown some examples to help our understanding. We watched a photogrammetry reel, which was very futuristic and cool and we also discussed the current market in VFX. Websites such as MyMiniFactory, Poliigon and Megascans have gained a lot of traction for selling printable 3d models & designs and are particularly popular with Blender and Unreal users.
The next 15 minutes of the lesson involved independent research. We were asked to conduct a case study on a particular piece of production that utilised this technique. Using articles that referenced Photogrammetry, we answered questions on how and why it was used. This was actually surprisingly difficult to find clear information on; I had to do a lot of digging and wanted to choose a unique film to research, but I ended up having to go with something already talked about before since there were hardly any useful sources of information on the topic. Furthermore, the sit that I was using was pretty unclear and only had a small paragraph on Photogrammetry. I did what I could to fill the gaps, using my own knowledge of The Matrix, which is the film I had chosen.
From extra research other than the above, I also found out that it can be used for really quick environments by scanning in a rock or tree. Additionally, it is sometimes used in museums in order to preserve assets that are on display and are handled a lot by the public. By having a 3D, digitalized version, there is always a backup in case of damage.
Capture
Now back to how photogrammetry works
In order to create a realistic 3D model, the more data inputted the better. This is why when taking photos of an object, it is best to capture it from various heights and angles. Every conceivable point of view is valid and useful because there is more overlap between images which ensures that the software has data of every perspective. Ideally, the images should be taken in RAW format and flat, even lighting where possible. Markers and grids are often used in the scene near the model to support overlap and alignment.
Of course, with one form of media capture, there is always the question of is there an easier, faster way? Well, there is, however it lacks the accuracy that taking many pictures has. This alternative method is to record a video orbiting around the object. A high framerate and narrow aperture are essential in order to avoid motion blur and keep the video in focus. Once it is recorded, it also needs to be converted into an image sequence for the effect to work.
Processing
Oftentimes, the next stage is necessary to remove any human error and flaws in the images (like touch-up). Processing them involves converting them from RAW to PNG, removing marks and dirt and adjusting the colour and contrast settings. Whilst this can make things easier, ideally, you should shoot and edit your photos to support the alignment and texture generation phase as much as possible.
Reconstruction
The final aspect is when the magic happens. Reconstruction is when the photos are transformed into a 3D model which is a process handled mostly by the software.
- Alignment
- Depth Extraction
- Reconstruction
- Texturing
Software:
- Meshroom (FREE)
- Reality Capture
- Metashape
We wanted to see what kind of results this generates and how they would look in the program we’re most familiar with. Using MyMiniFactory, we downloaded one of the models and imported it into Maya in order to review certain aspects and consider our own modeling capability in comparison. I had some trouble with the website initially since the first two downloads didn’t work at all, but on the third attempt, the model appeared.
It seemed that I had chosen a strange decorative bowl or lid of some kind. Originally, it was green and the indentations on top looked like leaves but of course, this didn’t carry across into Maya. It was incredibly complex and without even panning around it I was amazed to see all of the nooks, crannies, holes and cracks that were interpreted to precise detail. In ‘heads up display’, I turned on the poly and ngon count to see what the topology was like, and unsurprisingly, the numbers were massive!
Without even looking at the table in the top right, you can see how incredibly dense the model was. But for reference, it consisted of 998140 polygons, 998078 of which were ngons. This is quite literally a modeller’s worst nightmare since such complex geometry is incredibly difficult to work with. 99% of the model was made up of ngons and with anything that complex, trying to re-topologize in Maya would make it crash instantly.
If I were to model this, I would simplify the design greatly. Unless the object is for a museum or has some other purpose to be presented with such detail and accuracy, it doesn’t need so many bumps and cracks (we’re not just talking about the main ones here, for the decor, but the individual tiny little ones in between that make up the rough texture). To make it more usable as a simple, clean background asset, I would create it more symmetrical and reevaluate how to add the leaf pattern on top. Everything would be more minimal and easy on the eyes. However, if I was to attempt to model something like this (which I can’t at my current level) it would take me months to complete. The surface is so elaborate, that upon looking closely, I was reminded of the surface of a craggy landscape or planet!
So far, photogrammetry seems a very interesting technique and I am looking forward to learning more in the next lesson.