Considered Tracking Methods
For intermedia experiences, I considered three possible technical methods to track the paintings and play animations on the screen of the handheld AR device: fiduciary markers, Stb tracker markers, and Qualcomm AR SDK for Unity. Using fiduciary markers is the common way to provide a link to the AR device. The fiduciary markers next to the paintings and installations will help viewers track the art piece and activate animations. By scanning the fiduciary mark of each work, visitors will enjoy the animation related to the paintings on a wall and an installation. There are, however, several possible problems with using the markers. It is hard to find exact positions of the paintings and placing animations on the top of the paintings. Also, markers have to be seen on the screen all the time to avoid losing animations. If users get closer to see details of the painting, they will lose the animations on the screen. In addition, the black and white markers could harm the aesthetics of the paintings and installed artwork.
To solve/minimize these problems with using regular fiduciary markers, I considered using Stb tracker markers, which are like square frames. The paintings can then be placed inside of the frame-like markers. These markers define the area of playing animations. If users move their AR devices such that the paintings are out of the screen, the animations on the screen will automatically disappear. If the users move the devices back to the paintings, the animations will reappear. One possible problem is that users have to see the whole painting with the markers on the screen, which requires being some distance away. In short, users cannot get up close to see the details of the paintings and enjoy the animations on the screen at the same time.
My final solution was using Qualcomm Augmented Reality SDK for Unity 3D, viewable on Android smart phones. This AR technology also has a problem in that it is available only on Android phones and doesn’t support videos. However, it provides amazing image tracking without markers. Making paintings trackable requires several steps: scanning or taking photos of the paintings, importing the .jpeg or .png images on the website of Qualcomm AR, and exporting the asset files that contain codes and all trackable images. Then, the created files need to export to Unity 3D, a game development software program. In Unity 3D, I was able to create animations with transparent backgrounds by importing more image files on the top of the trackable images and writing codes. To generate animations in Unity 3D, I had to adopt stop-motion animation techniques. For example, I drew the children’s running movements frame by frame with a pencil. After scanning the hand drawn images, I got rid of the background of each drawing and adjusted the colors in Photoshop. Then, I imported the png files to Unity 3D to make the AR animations. In Unity 3D, I adjusted the sizes and spaces of the images, and I made the images play one by one by inserting a code. While researching three AR technology, I first tried the Qualcomm AR SDK for Unity 3D to create more immersive intermedia experiences, and the trial was successful.
(Unity 3D coding help from Richard Shemaka (Georgia Tech MS Student), Thank you.)
For intermedia experiences, I considered three possible technical methods to track the paintings and play animations on the screen of the handheld AR device: fiduciary markers, Stb tracker markers, and Qualcomm AR SDK for Unity. Using fiduciary markers is the common way to provide a link to the AR device. The fiduciary markers next to the paintings and installations will help viewers track the art piece and activate animations. By scanning the fiduciary mark of each work, visitors will enjoy the animation related to the paintings on a wall and an installation. There are, however, several possible problems with using the markers. It is hard to find exact positions of the paintings and placing animations on the top of the paintings. Also, markers have to be seen on the screen all the time to avoid losing animations. If users get closer to see details of the painting, they will lose the animations on the screen. In addition, the black and white markers could harm the aesthetics of the paintings and installed artwork.
To solve/minimize these problems with using regular fiduciary markers, I considered using Stb tracker markers, which are like square frames. The paintings can then be placed inside of the frame-like markers. These markers define the area of playing animations. If users move their AR devices such that the paintings are out of the screen, the animations on the screen will automatically disappear. If the users move the devices back to the paintings, the animations will reappear. One possible problem is that users have to see the whole painting with the markers on the screen, which requires being some distance away. In short, users cannot get up close to see the details of the paintings and enjoy the animations on the screen at the same time.
My final solution was using Qualcomm Augmented Reality SDK for Unity 3D, viewable on Android smart phones. This AR technology also has a problem in that it is available only on Android phones and doesn’t support videos. However, it provides amazing image tracking without markers. Making paintings trackable requires several steps: scanning or taking photos of the paintings, importing the .jpeg or .png images on the website of Qualcomm AR, and exporting the asset files that contain codes and all trackable images. Then, the created files need to export to Unity 3D, a game development software program. In Unity 3D, I was able to create animations with transparent backgrounds by importing more image files on the top of the trackable images and writing codes. To generate animations in Unity 3D, I had to adopt stop-motion animation techniques. For example, I drew the children’s running movements frame by frame with a pencil. After scanning the hand drawn images, I got rid of the background of each drawing and adjusted the colors in Photoshop. Then, I imported the png files to Unity 3D to make the AR animations. In Unity 3D, I adjusted the sizes and spaces of the images, and I made the images play one by one by inserting a code. While researching three AR technology, I first tried the Qualcomm AR SDK for Unity 3D to create more immersive intermedia experiences, and the trial was successful.
(Unity 3D coding help from Richard Shemaka (Georgia Tech MS Student), Thank you.)