Please NOTE that this documentation will not be updated anymore. Please see our new Developer Documentation Site for the latest updates! The content of this page can be found here.
|
The Basics
Development Questions
User Behaviors and Experience
The Basics
What is Layar Vision?
Layar Vision uses detection, tracking and computer vision techniques to augment objects in the physical world. We can tell which objects in the real world are augmented because the fingerprints of the object are preloaded into the application based upon the user’s layer selection. When a user aims their device at an object that matches the fingerprint, we can quickly return the associated AR experience.
What is an Augment? A “target object”?
An augment is a digital model (2D or 3D) rendered on a target object as seen through the mobile phone camera lens.
A target object is a physical or digital visual entity that contains enough detail to be unique whereby a person can point their phone at said “object” to see the augment (if available). Examples and best practices for creating a target object can be found in Reference Image Best Practices .
How do I integrate Layar Vision into a Layer?
It is simple to integrate Vision into your Layer. You need to upload images of your target objects, format your JSON response to include those images, their associated augments and actions, and publish. The API documentation can be found here.
Which products will get Layar Vision?
Layar will introduce Layar Vision in the iPhone and Android versions of the Layar Reality Browser app. A new iPhone Layar Player with Layar Vision will be released in Q4 of 2011. Finally, we are introducing an Android Layar Player with Layar Vision. This will be the first release of an Android SDK from Layar.
When will these products be launched to the public?
The Android Beta Layar Reality Browser and supporting platform updates will be available to Layar Developers in the coming weeks as a Public Beta. The iPhone and Android Layar Reality Browser apps will be available to the public in the app stores near the end of Q3 2011. The Layar Player (Android and iPhone) will be available soon.
Will Layar store the reference images?
A reference image is the file uploaded by a developer for analysis so that the object depicted in the file can become a target object; thereby, enabling target tracking of said object. Layar will extract the details of each image to create the unique fingerprint of the object. We will keep the reference image in our database as they might be needed for future improvements of our algorithms.
How will Layar use my reference images?
Layar uses them to extract fingerprints. You remain the owner of your reference images but Layar owns the fingerprints and reserves the rights to use them in all its products.
I am developing a layer for non-commercial use (non-profit, artistic or educational project). Is it possible to get a discount on Layar Vision usage pricing?
For non-profit, artistic and educational project pricing, please contact us at business@layar.com. In your request, please include a description of your project, as well as your contact details and the organisation you represent. You will receive a response from our business team shortly.
Development Questions
Layer Definitions Requirements:
The deltas for this release are to the minimum API version, Layer Type, and Layar Vision tab. The Mininum API Version: 6.0. The Layer Type: 3D and 2D objects in 3D space is required. In order to create an immersive and meaningful experience on objects, we only support 2D images and 3D models on target objects. You can still use CIWs and Default Icons on GeoPOIs. Finally, we added the Layar Vision tab where you will upload your reference images so that we may extract fingerprints for recognition.
For details on how to configure your layer in the Publishing Environment, see Enable Layar Vision in a layer .
Can I convert my older Layers into Layar Vision enabled layers?
Yes, but you will need to update your response to us to use the new formatting of the API v6.0. We'll take some measures to convert the new API responses to the old format for clients with older versions, so you can safely convert all your layers to the new formatting once we launch the Android Beta Layar Reality Browser. See GetPOIs API v6 Changes for the deltas. The full new getPOIs API documentation will be provided when the public beta is released.
Do I need to upload reference images to Layar?
Yes. In order for us to apply our Layar Vision algorithms, you will need to upload reference images to a Layar repository. We use these to extract the fingerprint of the object so that we can detect, track and augment the object.
Can other layers use my reference images?
No, only you (or your designee if Publisher role is handed over) may use the reference image. The fingerprints extracted can only be used by the associated layer so other layers will not have access to your images.
How will I be able to upload reference images?
You will be able to upload images for each layer in the Publishing Environment. In the future, you may be able to bulk upload, but the exact details on this are not available yet.
What format/size do the reference images need to be when uploaded?
Image resolution must be at least 640 x 480 pixels, though larger images are accepted and encouraged. Accepted file types include .JPG, .PNG, and .GIF. File size may not exceed 10 MB.
Is there any limitation for Layar Vision for a specific handset? Processing power / Graphic capabilities?
We actively support Android 2.2 and above, as well as iPhone 3GS and 4, with iOS4.0 and above. Android 2.1 is not recommended due to performance issues.
Tracking quality is dependent on the camera quality and lighting conditions. While we do have elements in place to account for variations in lighting, your camera may cause suboptimal exposure in very dark or light conditions. We cannot control this function in the phone.
Fluidity of tracking when in motion is affected by processing power. If your phone has limited processing power, you may see a lag in how quickly the augment moves as you (or the target) move.
What is the maximum number of target objects that are loaded from one request?
At present, we load a maximum of 50 target objects per call. We continuously work to improve that number and must account that not all phones are equal.
We are maintaining our current limit of 50 augments per response. If the developer sends back more than 50, we ignore the additional augments. This will apply for all augment types. The augments include those on objects and GeoPOIs. At this time, we will not differentiate between the two in counts. For performance reasons, we cannot increase this number yet, however, as we improve performance and processing of target objects, we will look into increasing the number. If the developer formats their response to include one augment per target object and does not include GeoPOIs, they can have the maximum 50 target objects. In summary, the limit applies to number of augments and is dictated by performance.
Do I need a special marker for Layar Vision?
No, this is not necessary. We encourage you to use real-world objects to create your feature sets, with the guidelines set in this document and on Reference Image Best Practices .
What makes a good reference image?
Be sure that your reference image meets the following criteria:
- Image does not contain reflections or transparency.
- Object in image is clear with no motion blur.
- Image contains the complete object, so no occlusions or crops unless it is a dynamic part of the object (such as text).
- Sufficiently high resolution of the object (i.e. not too small in taken image).
- Sufficient and consistent lighting. Be sure that object is not shadowed.
- The image is a frontal view of the object.
We provide a quality gauge in the Publishing Environment when you upload an image. Additional tips can be found on Reference Image Best Practices .
Will Layar Vision work on every object?
It will work on many objects, but there are limitations to what the technology is able to recognize and track. Layar Vision works best on planar surfaces, such as posters, billboards, flyers, book covers, CD covers, paintings, etc. The surface has to have a minimum amount of discriminative texture and details. Very simple/stylized logos, mainly uniformly coloured surfaces, repetitive patterns or text will be hard to track. The higher the contrast of features, the better.
Are there any surfaces or details of an object to avoid in order for Layar Vision to work, such as metal surfaces with carved textures or colors?
We recommend against transparency and reflectiveness as the results will not be of high quality and cannot be ensured. 3D items like sculptures are not suitable as target objects unless the surface to be detected is fairly flat.
Can there be variation between objects and submitted reference images? If so, how much? Example: variations of colors, contrast,changed text, additional icons.
For best results, the short answer is no. You will want to capture the object in the state that you expect your users to find it.
We advise that text is not the focal point and the only defining characteristic(s) of your reference image. You should only include text in your reference image if you expect the text to remain consistent in the real world.
A NOTE on color: currently, our algorithm is color agnostic, meaning the application does not see the difference between blue and green. Instead, what matters is that the luminosity and depth of the colors are significant. Those need to be high contrast and sharp.
Can you show a different augmentation depending on the user's location ?
Yes. In the getPOIs request, we send the user's the current location to the layer developer's web service. Based on this, you can modify your response to send the augment that you want to display for that particular location.
Does Layar Vision support video playback on top of an object?
Currently, we do not support video embedded on top of the image. However, you can link your Layar Vision object with a video as an action, which will invoke the device's native video player to play the video, giving the user a full screen mode of your video. For more information, please read Here. A tutorial on how to include actions to a layer is also available.
User Behaviors and Experience
What is the maximum angle a user can hold the phone from the target object?
The maximum threshold is 45 degrees in any direction. We train for the flat (parallel) use case; however, we do have some training for reasonable angles. The threshold will improve over time; but the current conditions are optimal for data collection, device limitations, and processing time.
Can I use an image to open a layer?
No. The way we implement Layar Vision doesn't allow the user to use Layar as a scanner. We will not make major changes to the user experience. Users will need to open a layer in the usual way, e.g. by browsing the layer catalog, or by reading a QR code (containing a Layar intent) with a third-party scanner. Once the user is "within" a layer, the application will act as a scanner, looking for target fingerprints to track.
How robust is detection of target object under different lighting conditions?
We normalize for lighting variations. Of course, taking a picture in the closet is not going to make a difference.
Can the target object be rotated up to 360 degrees, or does it have to be detected in a single correct orientation?
Yes, you can spin the object. The augments will spin with it.
How large does the detected target object need to be?
At a minimum, the object needs to encompass about a quarter of the screen for best results.
Will there be a detection reticle displayed on screen to guide the user to frame the image? Or will detection work no matter where the image is displayed on the screen?
No, detection of the object works no matter where the image is displayed on the screen.
How long and steady does the user need to hold the device over the target object for detection to happen?
When you move, the augment moves. The higher resolution phones will have a great sensitivity.
Does the entire target object need to be visible for detection to work? Or can some parts be occluded or offscreen?
When you (or the object) moves, the augment will move. Once we lose tracking completely, the augment will "fall off" the screen. The percentage needed to track is dependent on the image.
Would target objects be recognized if they were displayed on a screen (TV, computer etc?)
Yes and no. Computer screens are fine (in house testing was done on a Mac laptop screen). TV and movies are poor choices for showcasing target objects as, by definition, the object is constantly in motion.
Comments (0)
You don't have permission to comment on this page.