Fieldwire Engineering

Background

Fieldwire has a 360° photo feature for our users to capture and record a 360° view of their jobsite. These 360° photos can be added to projects, and then viewed on our web, Android, and iOS apps.

On Android, we have been using the Google Play Services’ Panorama API to display the 360° photos. Unfortunately, that API stopped working on Android 11. We suspect that it is due to the introduction of Scoped Storage on Android 11, and the Panorama library not being updated to handle the new requirements. Regardless of the cause, it is an issue that is affecting other Android developers as well, with no workaround that we could find.

Our iOS app has been using a different library by Google - the Google VR SDK, which they have recently replaced with an in-house solution that they built from scratch. We decided to migrate to the Google VR SDK on Android because this was already a production issue affecting our real users, and using that library was the quickest solution available to us at the time.

Initial solution

The first iteration of our solution involved using a VrPanoramaView and calling its loadImageFromBitmap method with the 360° photo bitmap that has been loaded into memory:

    
    override fun onBitmapFetched(imageUrl: String, primaryView: V, bitmap: Bitmap) {
        // hide our progress indicator
        progressManagingCallback.onBitmapFetched(imageUrl, primaryView, bitmap)

        // pass the bitmap into the VrPanoramaView
        vrView.loadImageFromBitmap(bitmap, null)
    }

This gave us a 360° photo viewer, with the following characteristics:

Starts off in the gyroscopic mode (where the user changes the viewing angle by physically turning their device)
The gyroscopic viewing mode allows the user to touch-scroll horizontally, but not vertically
Has no way for the user to switch to a pure touch-scrolling mode
Has a button to toggle Cardboard mode and an Info button that will open a webpage that explains VR view
Has a button to toggle fullscreen mode
Does not support zooming

It is a “functional” 360° photo viewer, but there are a few tweaks we needed to make in order to ensure an experience similar to what we were previously providing.

Disabling fullscreen and Cardboard modes

First thing’s first, we didn’t need or want to support Cardboard mode at this stage. This also made the Info button obsolete. Furthermore, the fullscreen mode had no value to us because we already made the VrPanoramaView span the entire screen. Fortunately, the VrPanoramaView has a few methods we could call to hide those buttons.

    vrView.apply {
        setFullscreenButtonEnabled(false)
        setInfoButtonEnabled(false)
        setStereoModeButtonEnabled(false)
    }

Viewing modes

Our original 360° photo viewer starts off in a pure touch-scrolling mode, with a button to toggle to a gyroscopic mode. It also does not allow for touch-scrolling when in gyroscopic mode. This was the default behavior of the Play Services Panorama viewer we were using before.

The VrPanoramaView we are now using has methods to control both behaviors:

    btnVrMode.setOnClickListener {
        when (vrViewMode) {
            VrViewMode.PURE_TOUCH -> {
                vrViewMode = VrViewMode.GYROSCOPE
                vrView.setPureTouchTracking(false)
                vrView.setTouchTrackingEnabled(false)
            }

            VrViewMode.GYROSCOPE -> {
                vrViewMode = VrViewMode.PURE_TOUCH
                vrView.setPureTouchTracking(true)
                vrView.setTouchTrackingEnabled(true)
            }
        }

        updateVrToggleButton()
    }

The code should be mostly self-explanatory.

btnVrMode is simply an ImageButton we added on top of the VrPanoramaView
VrViewMode is an enum we created to track the current viewing mode of the viewer
updateVrToggleButton() is a local function to update the icon being displayed on btnVrMode based on the value of vrViewMode

We use the Android Iconics library for some of our icon resources, and these are the icons we are showing on btnVrMode based on vrViewMode:

    private fun updateVrToggleButton() {
        val icon = when (vrViewMode) {
            VrViewMode.GYROSCOPE -> GoogleMaterial.Icon.gmd_3d_rotation
            VrViewMode.PURE_TOUCH -> GoogleMaterial.Icon.gmd_touch_app
        }

        btnVrMode.background = IconicsDrawable(context, icon)
                .respectFontBounds(true)
                .color(Color.LTGRAY)
                .sizePx(context.resources.getDimensionPixelSize(icon_dimension))
    }

Note that these are not the exact same icons shown by the Play Services Panorama API that we were previously using, but I believe they are very good approximations for the meaning we’re trying to convey (touch mode vs gyroscopic mode).

Zooming support

Now the last bit of functionality we needed to add to the new viewer is the ability to zoom. This is the high level overview of how we did it:

Created a TouchInterceptingLayout class which extends FrameLayout
Place the VrPanoramaView inside the TouchInterceptingLayout
Override the onInterceptTouchEvent(ev: MotionEvent?): Boolean method in the TouchInterceptingLayout where we pass the MotionEvent to a ScaleGestureDetector
Create a ScaleGestureListener class which extends SimpleOnScaleGestureListener, and use that as the listener for the ScaleGestureDetector mentioned above
In the ScaleGestureListener, override the onScale(detector: ScaleGestureDetector): Boolean method to use the value of detector.scaleFactor to scale the VrPanoramaView to implement a zooming behavior when the user performs a pinch-zoom gesture

My next blog post will go into more detail about how the TouchInterceptingLayout interacts with the VrPanoramaView (which is itself a FrameLayout containing other views and intercepting touch events itself), and how to apply the scaling effect on the VrPanoramaView.

Building a 360° Photo Viewer in Android 11

Background

Initial solution

Disabling fullscreen and Cardboard modes

Viewing modes

Zooming support