Stereoscopic 3D

Stereoscopic 3D Sample Code

The following code snippets based on the S3D demos illustrate the ability to enable or disable the Stereoscopic 3D display for various scenarios. The complete source code is available as part of the SDK download.

S3D Demos

S3D Demos main list activity:

S3D Image Demo

The first demo is the S3DImageActivity:

A test image provided as a resource drawable. Note that this file is named test.jpg with no extra metadata, so we have to programmatically turn on S3D to view in 3D.

From S3DImageActivity.java:

protected void onCreate(Bundle savedInstanceState{
    super
.onCreate(savedInstanceState);
    
setContentView(new MySurfaceview(this));
  

For this example we will not use a xml based layout resource. We will create our own View, a SurfaceView that we will define as a nested class.

class MySurfaceview extends SurfaceView implements SurfaceHolder.Callback {
        
private Canvas mcanvas;
        private 
Bitmap mbitmap;
        private 
Surface msurface;
        private 
SurfaceHolder mholder;

        public 
MySurfaceview(Context context{
            super
(context);

            
mbitmap BitmapFactory.decodeResource(getResources(), R.drawable.test);

            
mholder this.getHolder();
            
mholder.setFormat(PixelFormat.RGBA_8888);
            
mholder.addCallback(this);

            
msurface mholder.getSurface();
        

Note that we will implement the interface, SurfaceHolder.Callback The reason is two-fold, we want to be able to set the format of the surface to be RGBA_8888, noting that there needs to be an alpha channel and second we want to lock access to the canvas as we're drawing on it since enabling S3D requires a surface and we wouldn't want two threads setting the S3D format on the same surface.

public void surfaceChanged(SurfaceHolder holderint formatint widthint height{
      boolean formatResult 
true;
      
this.width  width;
      
this.height height;
      
      try 
{    
        formatResult 
DisplaySetting.setStereoscopic3DFormat(surface,
                  
DisplaySetting.STEREOSCOPIC_3D_FORMAT_SIDE_BY_SIDE);
      
catch (NoClassDefFoundError e{
        android
.util.Log.i(TAG"class not found - S3D display not available");
        
s3DAvailable false;
      
}

      android
.util.Log.i(TAG"return value:" formatResult);
      
      if (!
formatResult{
        android
.util.Log.i(TAG"S3D format not supported");
        return;
      
}

      canvas 
holder.lockCanvas();
      if (!
s3DAvailable{
        
// if DisplaySetting not found, create anaglyph version of 3D image
        
bitmap BitmapUtils.createAnaglyphBitmap(bitmap);
      
}
      canvas
.drawBitmap(bitmapnull, new Rect(0,0,width,height), null);
      
holder.unlockCanvasAndPost(canvas);
    

This is the important callback of the SurfaceHolder.Callback because we can't set the S3D format until we have a surface. Note that we need to check the result of our call to DisplaySetting.setStereoscopic3DFormat in case the method fails for any reason. It will return true if it succeeded setting the format otherwise false.

If it should fail, you will be displaying a side be side LR image, which is probably not what you intended. In this case you may want to crop the L or R image out and scale the image to the appropriate size.

Supporting devices that don't have S3D including the emulator

Currently, if the class com.htc.view.DisplaySetting is not available you will get a NoClassDefFoundError when you run the line in a device that is not a HTC S3D supported device.

result = DisplaySetting.setStereoscopic3DFormat(msurface, DisplaySetting.STEREOSCOPIC_3D_FORMAT_SIDE_BY_SIDE);

To avoid this, you can do one of the following:

  • Add a try-catch block handling the NoClassDefFoundError error (as shown in the example). note: additional error handling is omitted for brevity see source code.
  • Or check ahead of time to see if the class exists using Class.forName
  • Or wrap the DisplaySettings API in a compatibility class that checks internally using Java reflection API, returning false from the method for unsupported devices. An example is included The example also provides an additional bitmaps utility class that demonstrates how to fall back to the anaglyphic 3D (red-cyan) type of S3D with glasses to support non S3D devices such as the emulator.

Emulator support (red-cyan glasses required):

Note: All the 3D anaglyphic code provided with the demos are provided only part of the sample code and not natively supported by the emulator or the SDK officially.

Lastly, the onTouchEvent(MotionEvent event) illustrates how to toggle the S3D display settings from 3D back to 2D

S3D OpenGL Demo

The S3DOpenGLActivity activity demonstrates how to code OpenGLES (or port existing projects) to use the S3D display.

As previously shown in the image demo, we just need to use the DisplaySettings method to enable or disable S3D. But like the image demo, we need to place the left and right images of every frame that we draw side by side. This requires drawing the opengl world twice filling the left and right screen equally, as shown below. Also note that both left and right are compressed horizontally (like with the image demo) to comprise the full image ratio when overlaid in S3D mode.

This demo consists of drawing a spinning 3D cube using OpenGLES and additionally allowing user interaction via touch to spin the cube, zoom in and out using a seek bar control and a button to toggle between 2D and 3D. These 2D controls are overlaid in the XML layout definition and are thus separate from the OpenGL based SurfaceView which will be S3D enabled.

S3D OpenGL demo in the emulator with S3D enabled (using anaglyphic fallback) and with S3D disabled:

Some basic understanding of OpenGLES on Android is assumed but as a quick review, the 3D object, the cube or box is defined in Box.java, a GLSurfaceView is extended and provided in S3DOpenGLSurfaceView.java which also implements GLSurfaceView.Renderer. The main activity, S3dOpenGLActivity will be initialized the components defined in the layout, namely the GLSurfaceView, the Button and the SeekBar. There is one texture which is loaded as an image resource.

In S3DOpenGLActivity:

protected void onCreate(Bundle savedInstanceState{
        super
.onCreate(savedInstanceState);
        
setContentView(R.layout.main);
        
glSurfaceView = (S3DGLSurfaceViewfindViewById(R.id.glview); 
        
s3dButton = (ButtonfindViewById(R.id.button1);
        
s3dButton.setText("S3D\nis  ON");
        
myZoomBar = (SeekBarfindViewById(R.id.zoomBar);
        
myZoomBar.setProgress(3);
        
myZoomBar.setOnSeekBarChangeListener(myZoomBarOnSeekBarChangeListener);
        
setZoomLevel();
    

In S3DOpenGLSurfaceView we set up the surface view and opengl:

private void init() {
    holder 
getHolder();
    
holder.addCallback(this);
    
holder.setType(SurfaceHolder.SURFACE_TYPE_GPU);
    
this.setRenderer(this);
    
this.requestFocus();
    
this.setFocusableInTouchMode(true);    
    
box = new Box();
  
}
  
public void onSurfaceCreated(GL10 glEGLConfig config{    
    box
.loadGLTexture(glcontext);
    
gl.glEnable(GL10.GL_TEXTURE_2D);
    
gl.glShadeModel(GL10.GL_SMOOTH);
    
gl.glClearColor(0.5f0.5f0.5f0.5f); //background color
    
gl.glClearDepthf(1.0f);
    
gl.glEnable(GL10.GL_DEPTH_TEST);
    
gl.glDepthFunc(GL10.GL_LEQUAL);
    
gl.glHint(GL10.GL_PERSPECTIVE_CORRECTION_HINTGL10.GL_NICEST); 
  

The draw methods:

  • draw(gl) is the main OpenGL draw call and is called by onDrawFrame(gl) which will manage drawing the left and right views for S3D.
  • draw(gl) draws the basic world and it's objects and can be existing code from existing OpenGL application, however we'll move frame initialization such as clearing the screen to onDrawFrame(gl)
public void draw(GL10 gl{
    gl
.glTranslatef(z0.0f0.0f); // zoom
    
gl.glScalef(0.8f0.8f0.8f);
    
    
gl.glRotatef(xrot0.0f0.0f1.0f);
    
gl.glRotatef(yrot0.0f1.0f0.0f);
        
    
box.draw(gl);

    
xrot += xspeed// current spin on x and y
    
yrot += yspeed;
  

In onDrawFrame(), the way we'll draw L and R is by defining view ports into the world and by using the gluLookAt() function, which takes an eye position, a position to look at, and an up vector in object space coordinates to allow correct perspective for each eye's view. Consequently the distance between the eyes are to be taken to account for the S3D effect desired. Another important call is glFrustrumf() which describes a perspective matrix that produces a perspective projection which is different for each eye.

public void onDrawFrame(GL10 gl{
        float left
,right,top,bottom;
        
        
gl.glClear(GL10.GL_COLOR_BUFFER_BIT GL10.GL_DEPTH_BUFFER_BIT);
        
gl.glColorMask(truetruetruetrue);

        
//LEFT
        
gl.glViewport(00, (int) width 2, (int) height);  
        
gl.glMatrixMode(GL10.GL_PROJECTION);
        
gl.glLoadIdentity();
        
left   =   (float) (- ratio wd2 0.5 camera.eyeSeparation ndfl);
        
right  =   (float) (ratio wd2 0.5 camera.eyeSeparation ndfl);
        
top    =   wd2;
        
bottom = - wd2;
        
gl.glFrustumf(left,right,bottom,top,near,far);

        
gl.glMatrixMode(GL10.GL_MODELVIEW);
        
gl.glLoadIdentity();
        
GLU.gluLookAt(gl,camera.viewPos[0] r[0]camera.viewPos[1] r[1]camera.viewPos[2] r[2],
                  
camera.viewPos[0] r[0] camera.viewDirection[0],
                  
camera.viewPos[1] r[1] camera.viewDirection[1],
                  
camera.viewPos[2] r[2] camera.viewDirection[2],
                  
camera.viewUp[0]camera.viewUp[1]camera.viewUp[2]);
        
draw(gl);
        
        
//RIGHT
        
gl.glViewport((int) width 20, (int) width 2, (int) height);
     
        
gl.glMatrixMode(GL10.GL_PROJECTION);
        
gl.glLoadIdentity();
        
left   = (float) (- ratio wd2 0.5 camera.eyeSeparation ndfl);
        
right  = (float) (ratio wd2 0.5 camera.eyeSeparation ndfl);
        
top    =   wd2;
        
bottom = - wd2;
        
gl.glFrustumf(left,right,bottom,top,near,far);
    
        
gl.glMatrixMode(GL10.GL_MODELVIEW);
        
gl.glLoadIdentity();
        
GLU.gluLookAt(glcamera.viewPos[0] r[0]camera.viewPos[1] r[1]camera.viewPos[2] r[2],
                      
camera.viewPos[0] r[0] camera.viewDirection[0],
                      
camera.viewPos[1] r[1] camera.viewDirection[1],
                      
camera.viewPos[2] r[2] camera.viewDirection[2],
                      
camera.viewUp[0]camera.viewUp[1]camera.viewUp[2]);
        
draw(gl);
     

The calculations and math utilities that determine the parameters for glFrustrumf() and gluLookAt() are found in the Camera class which also determines the eye separation. Notes: 1) these calculations are provided as reference sample code and can be improved upon performance-wise as well as use different algorithms such as shifting the symmetric viewing frustum to adjust the parallax effect. 2) for clarity, anaglyphic fallback mode was omitted above, see the source code for full listing.

S3D Video Demo

This activity demonstrates the ability to play a 3D video clip in your application, touching the screen will pause or play the video. The video needs to be encoded with the SEI frame packing bit set to LR SBS (see overview for information). Additionally, a text view is shown over the video if paused.

In S3DVideoActivity's onCreate:

public void onCreate(Bundle savedInstanceState{
        super
.onCreate(savedInstanceState);
        
setContentView(R.layout.video);
        
preview = (SurfaceViewfindViewById(R.id.surface);
        
text = (TextViewfindViewById(R.id.text);
        
holder preview.getHolder();
        
holder.addCallback(this);
        
holder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    

We place a SurfaceView in the XML based layout as well as a standard 2D text view that will overlay the video. Note the standard setup for the SurfaceHolder and video and that the activity implements OnPreparedListener, OnVideoSizeChangedListener, and SurfaceHolder.Callback.

The code contains standard MediaPlayer lifecylce code reading the 3D mp4 video from the assets folder. As in previous examples, the S3D is set on the DisplaySettings method, setSteroscopi3DFormat in the surfaceChanged callback.

public void surfaceChanged(SurfaceHolder surfaceholderint iint jint k{
      holder 
surfaceholder;
        
enableS3D(trueholder.getSurface()); //note SEI FPA flag in content overrides this
    

Where enableS3D() is a convenience method for toggling between 2D and S3D display modes:

private void enableS3D(boolean enableSurface surface{
    text
.setVisibility(View.INVISIBLE);
    
int mode DisplaySetting.STEREOSCOPIC_3D_FORMAT_SIDE_BY_SIDE;
    if(!
enable{
      mode 
DisplaySetting.STEREOSCOPIC_3D_FORMAT_OFF;
    
}    
    boolean formatResult 
true;    
    try 
{
      formatResult 
DisplaySetting.setStereoscopic3DFormat(surfacemode);
    
catch (NoClassDefFoundError e{
      text
.setVisibility(View.VISIBLE);
      
android.util.Log.i(TAG"class not found - S3D display not available");
    
}
    
if (!formatResult{
      android
.util.Log.i(TAG"S3D format not supported");
    
}
  } 

S3D Camera Demo

This activity demonstrates the ability to record a 3D video clip from your application using the standard MediaRecorder mechanism, but also with the ability to toggle the preview display (and subsequent recorded video) between 2D and 3D by toggling via onTouch(). The camera preview mode is shown as 2D or 3D via a text view overlay on the camera preview SurfaceView.

In S3DCameraActivity the following is what's unique to this example compared to the previous examples for enabling S3D camera preview:

public Camera getS3DCamera() {
    Camera camera 
null;
    try 
{
      camera 
Camera.open(CAMERA_STEREOSCOPIC);
      
camera.setPreviewDisplay(holder);
      
is3Denabled true;
    
catch (IOException ioe{
      
if(camera!=null{
        camera
.release();
      
}
      camera 
null;
    
catch (NoSuchMethodError nsme{
        is3Denabled 
false;
        
text.setVisibility(View.VISIBLE);
        
Log.w(TAG,Log.getStackTraceString(nsme));
    

    
return camera;

Where CAMERA_STEREOSCOPIC) is 2 for camera number 2 (note: 0 is the default back facing camera, and 1 is the front facing camera).

Note: both the camera preview and video demos are not available for executing on th emulator and as shown above, the text will show the default message of "not available"