Creating 3D Interactive Objects from Photographs

In late 2017, I was approached by Florent Poux, an adjunct professor at the University of Liège and CEO of Geovast 3D. Florent is a passionate about ‘reality capture’. He proposed to work together in creating 3D interactive objects from photographs I take in abandoned buildings. Shortly after, we met in person, and Florent took me through the whole process of creating a 3D object from photographs with photogrammetry and lasergrammetry.

Photogrammetry and lasergrammetry are part of what we call ‘reality capture’, which deals with creating the physical reality of objects and places. Our enthusiasm led us to immediately put our thoughts into action. We’ve been working on this project a lot in the past years. In this article, I would love to take you with us through the process and show you a few examples. At the same time, I will share parts of the technical side, some of the challenges we faced, and the numerous possibilities this technique has to offer.

What is photogrammetry?
Photogrammetry is the reconstruction of ‘object space’ from ‘image space’, or the reverse process of photography. Among other things, photogrammetry enables you to measure with photos.

What is lasergrammetry (or laserscanning)?
Lasergrammetry is a method in which a laser is used to scan a surface. This laser is emitted in all directions to scan the surrounding space, and returns the precise position of millions of 3D points for each spot. This allows us to retrieve information about the shape and characteristics (such as reflectance of the surface) of the real-world environment and convert it into a digital object.

If you’re interested in learning more about this, or if you would like to work on a project together, feel free to contact me. Check out our YouTube channel for more examples.

The process and technique

The video above shows a finished example of one of our first attempts at creating 3D interactive objects from photographs. This video is created solely from taking photographs (photogrammetry) in a room in Rocchetta Mattei. Compare the video with a regular photo of The Lions’ Courtyard in Rocchetta Mattei to recognize the differences.

Rocchetta Mattei

Rocchetta Mattei

The room is located in an amazing castle in Italy that is called Rocchetta Mattei. Built in the mid-19th century, it used to be the home of count Cesare Mattei, a self-taught physician who founded electrohomeopathy, a practice stemming from homeopathy. During World War II, German troops damaged the interior of the building. When the war was over, the last heir could not sell the building and offered it free to the Municipality of Bologna. They did not accept the donation. In 1959, Primo Stefanelli purchased Rocchetta Mattei. He transformed one of the smaller buildings into a cozy hotel with an adjoining restaurant. In 1989, Stefanelli died and the building deteriorated. Rocchetta Mattei was closed to the public. In 2006, the foundation of Cassa di Risparmio in Bologna announced the acquisition of Rocchetta Mattei. They restored it, which ultimately led to the reopening to the public on the 9th of August 2015.

In the overview below, you can see the positions and angles I used for my photographs involved in the creation of the finished product (the video). I created a circle in the room and took photographs. I started in one position, made three shots from different angles (middle pointing forward, low pointing up, and high pointing down) and then moved a step to my right. I repeat these steps until the circle was complete. After that, I made a few close-up shots of the fountain to be able to extract the high level of detail (at a millimeter precision) in post-processing, which you can clearly see in the video.

Keeping the focal length and exposure exactly the same in each shot is of the utmost importance, because the photogrammetric processes extract key features from each photo. Afterwards, these photos are matched with other overlapping photos. If the radiometric conditions are too different, this process fails.

The entire video was created with 168 photos. The software we use automatically matches the position of each photo and combines them into a 3D object. Because I’ve mainly focused on photographing the fountain, you might have noticed black spots in the corners of the room. Those black spots lack data because I didn’t photograph those areas well enough. There is no digital information to appear in the frame.

In the ‘Flyvast Version‘ (Flyvast is 3D web point cloud data processing software), we have placed the fountain as a 3D interactive object. This allows you to zoom in on details of the object, interactively walk around the object, label elements, and perform real-life measurements. Since I did not photograph the top of the fountain, there is no sufficient data to show the top as a whole. Nevertheless, the fountain is semi-perfect and could easily be printed as a 3D object, for example.

Apart from Rocchetta Mattei, we also did a project in a beautiful rococo library in Rolduc Abbey in The Netherlands and a former wool washing facility in Belgium.

Both of the projects involving the creation of 3D interactive objects from photographs differ from the first project, because we have used lasergrammetry in addition to photogrammetry here. The added BLK360 laser scanner by Leica Geosystems emits around 1 million pulses per second. This precisely records the environment in the form of a very dense point cloud. Lasergrammetry is different from photogrammetry because of the sensor and underlying approach. Lasergrammetry emits light (active sensor) and photogrammetry uses passive sensors that measure the light received (e.g. photo camera). The processes to merge different perspectives in one common 3D model is then based on common points, either extracted automatically or not.

Combining lasergrammetry with photogrammetry is suggested but definitely optional. Lasergrammetry gives exceptional precision, and it is simple to assess the quality. However, it suffers from poor radiometric coverage. Photogrammetry, on the other hand, is much more flexible as you can move around and take shots from every angle, thus refining the base model. It also allows you to obtain texture with a full range of freedom for an artistic or realistic feel.

Lasergrammetry can work flawlessly in the dark and at really long distances. It can also create a full 360° digital interactive dome object.

 

Rococo library

Rococo library

In the year 1104, a young priest, Ailbertus from Antoing, appeared in the Land of Rode accompanied by two friars. They wanted to lead a simple life of absolute poverty. Adelbert, Count of Saffenberg on the Ahr and owner of the castle in ‘s Hertogenrade, offered them a strip of land on which they built themselves a simple home and a wooden chapel. Over the next 900 years, this wooden chapel transformed into the largest abbey complex of the Benelux and one of the Dutch UNESCO Top 100 monuments.

In the mid-12th century, the abbey flourished. In 1250, it possessed about 3000 hectares of land, and the its membership numbers increased steadily. The richly decorated Rococo library attracted an important collection of books and developed into one of the most important libraries of that time. The library today still contains books from as far back as the 13th century. During the Middle Ages, the Rolduc library was one of the most famous libraries in the Meuse region.

The ‘Flyvast version‘ of the Rococo library shows the incredible amount of detail we’ve been able to capture. However, it also shows some of the challenges faced during the process of creating a 3D object like this. For example, if you zoom in on the piano, you can see that the instrument is not complete. That is because bright sunlight shone on the piano, and the surface of the piano reflected that light. The camera is able to capture this light and detail, but the software is unable to convert it into a 3D object due to the impossibility to recover a ‘precise’ point of interest. This is because the pixels have different values (highly reflective) depending on the position of the photo camera.

In addition to a movie and interactive object, we are also able to create 3D overview images from objects or rooms. The images below are examples of what can be achieved.

The biggest project we have done thus far focuses on an abandoned wool washing facility in Belgium. We have captured multiple rooms and the exterior (low-quality) and connected them with each other in post processing. See the video below for an example of the result.

 

Wool washing facility

Workplace

Originally established in 1899, the facility used a new process, developed in the United States, to degrease and wash wool, whereby the by-products, such as wool grease, were recovered. The facility used the water from the river nearby, mainly in the steam engines. The river water is very pure and contains hardly any chalk. The facility cleaned the wool with hot water and detergent. The lanolin or wool grease floats on the water, is drained, filtered, and purified. Lanolin used to be a valuable product used in cosmetic products. Some of the steam engines were still in use till the end of the 20th century. Volunteers are now working hard to transform the abandoned site into a museum.

The facility itself is a beauty to photograph. I couldn’t resist sharing a few photos with you that I took on-site.

The ‘Flyvast version‘ gives you a lot of room to play around. You can virtually walk through the rooms that we’ve photographed and connected with each other. Because it was raining heavily while shooting for the project and the roof was partly broken, the rainwater caused reflections on some of the objects (the chair in the workplace for example).

The options available when doing a project like this are endless. Explore some of the most useful project types associated with the process of creating 3D interactive objects from photographs below.

How can this support you or your business?

This process and technique offers a variety of benefits for a personal or business related purpose.
Take a look at the following examples:

• Printed 3D object
  You are the proud owner of a vintage car and you would like to have a printed miniature of your car. We can photograph the car for you and 3D print a scaled version of your car, in any (reasonable) size you prefer.
• Virtual 3D tour
  You work in, or are the owner of, a building that has rooms that are closed for the public. However, these rooms have historical value and you would like to offer the public a chance to experience these rooms. We can create a detailed virtual and interactive 3D tour for people to view or virtually walk through on the internet.
• Extended reality (such as augmented- or virtual reality)
  You are, or want to become, an escape room owner and you would like to create an escape room that guests can play by putting on Virtual Reality glasses, or you would like to project a digital layer in reality. We can create the 3D foundation for you to create such an experience.
• Orthophoto (metric images)
  You are an operator and you would like to measure true distances of a surface (aerial or ground). We can create an orthophoto for you with an accuracy of 1 milimeter that allows you to carry out exact measurements, digitize or place linework, and place text annotations or geographic symbols.
• Restoration
  You are an architect working on a restoration project. We can create a highly detailed digital interactive 3D photo for you that you can use as a reference while working off-site. It will allow you to automatically extract an unlimited number of precise cuts and sections to support your new creative design.
• Gaming
  You are a game creator or designer and you would like to include 3D objects, rooms or buildings in your game. We can photograph objects, rooms and buildings and present them in the desired 3D format.
• YouTube video
  You are a creative blogger and you would like to showcase an object or room in a YouTube video. We can create a custom video for you with your desired path of movement, which includes speed, focus, and direction.
• Preservation
  You work in the historic preservation sector and you would like to preserve, conserve and protect buildings, objects or other artifacts of historical significance. Any of the options listed in this opportunity summary would be invaluable to your work.

If you’re interested in learning more about creating 3D interactive objects from photographs, or if you would like to work on a project together, feel free to contact me.