getUserMedia() Logs To The Rescue

Remus -

For a long time we've been in the dark about the problems and errors users had with granting access to camera and microphone devices. We knew there were occasional problems but we had no idea how often they were occuring, why, who was impacted and if they are avoidable.

More than 1 year ago we've taken taken the 1st step and tried to understand the type of problems a user could face. It was clear then there's no one simple answer.

Not anymore!

A few months ago we've started logging these errors server side in a half baked attempt to measure their occurence. It didn't took us long from that 1st attempt to better understand the data, put it in a database and make it available - in a usable form - to our clients in their account area.

So today we're introducing a new Logs section that gives you access to all the successful and failed attempts to access the camera and microphone with getUserMedia().

The data is gathered from our HTML5 desktop recorder's attempts to access the camera and microphone, including when it is used inline on Chrome on Android.

The new Logs section is a very useful tool.

  • It will help you and your customer support team identify problems with your users' cameras, microphone and permissions and answer their complaints
  • You'll be able to look up their attempts to start their camera or mic and find out whether they have a hardware problem or a permissions problem (dismiss or deny).
  • It will give you accurate analytics of how often your users interact with the permission dialog and how often they block device access or dismiss it.

This post will go over the entire new section and help you correctly interpret the data, so let's delve right in.

The gathered data

All the gathered data can be found in the new Logs section. To access it, click the new Logs tab in the Pipe dashboard:

Access to the data differs between account types:

  • Standard accounts have access up to 72 hours of data
  • trial and PRO accounts have access up to 30 days of data

From left to right you will see the following data columns:

  • Date
  • IPv4
  • Browser
  • OS
  • Cam
  • Mic
  • Status
  • Description
  • Recording errors
  • TTA (time to action)

What does each column contain?

Date & time

The date & time at which the log has been created (not the time of the getUserMedia() call) in the Pipe account time zone (so might not match the user's timezone)

IPv4

The IP address of the user's device. You can filter by IP by clicking on it or by using the search box located above. When a user emails you with a problem you can quickly see all their attempts to allow camera and mic access by filtering by their device IP.

There is an additional drop-down menu to filter by error type. We will talk more about those below.

Browser & OS

The Browser coumn gives you the browser & version.

In the OS column you'll see the usuals (Windows, macOS, Linux, etc.. ) and also Android if you have the inline desktop recorder activated for Chrome on Android.

Cam & Mic (permission ahead of request)

These 2 columns store the permission state of the camera and microphone BEFORE the actual getUserMedia() attempt to access the camera and/or microphone. This data is only gathered for attempts made on Chrome version 64 and up. Firefox and older versions of Chrome do not have the needed API to collect this data.

The possible values for the Cam and Mic columns are:

  • prompt: the user has not yet been asked to allow device access so when the browser will request access a permission dialog will be shown
  • granted: the user previously granted access to the device thus the permission dialog will not be shown. The device will be accessed without prompting the user
  • denied: The user previously blocked access to the camera and/or microphone, the permission dialog will not be shown and device access will be denied
  • - : no data is available, the log was created using a browser that does not provide the data (Firefox or an older version than Chrome 64)

Status (permission after request)

The Status column displays the result of the attempt to access the camera and/or microphone:

  • OK: the getUserMedia() attempt to access the camera and/or microphone was successful
  • NotAllowedError: the user has denied access to the camera and/or microphone at the browser or OS level or there was a permission related problem
  • NotFoundError: the user granted permission to use the camera and/or microphone, but no camera or microphone could be found
  • NotReadableError: the user granted permission to use the camera and/or microphone, but a hardware error occurred at the operating system, browser, or web page level which prevented access to the device
  • OverconstrainedError: the camera does not support the given resolution or the microphone cannot capture audio with the given parameters
  • AbortError: although the user and operating system both granted access to the hardware device, and no hardware issues occurred that would cause a NotReadableError, some problem occurred which prevented the device from being used

A legend with all the status types is also present at the bottom of the Logs page.

You can filter the logs by status type through the drop-down menu at the top. This can also be combined with IP filtering to get even more relevant results. In the example below I've made a search for all the NotAllowedError entries that were caused by a certain IP:

Description

The Description column contains additional data about specific error types that help you better understand the circumstances of the error. This data can either be:

  • A description for the getUserMedia() errors (the ones described in the Status column). For example, you can differentiate between a block and a dismiss on Chrome because a block will generate a NotAllowedError with the description Permission denied, while a dismiss will generate a NotAllowedError with the description Permission dismissed.
  • In the case of Recording errors this description will contain the camera and/or microphone names.

Recording errors

The Recording errors column stores errors that can be triggered AFTER a successful getUserMedia() attempt when recording:

  • Device error: no audio or video data: a catch all message for some rare corner cases which in the past produced empty recordings (0 Kb). We've been able to reproduce the problem in the following cases: removing your device just before recording, using an unsupported device on macOS, not allowing OS level device permissions on macOS the 1st time you're asked, other apps having exclusive access to the device on Windows and recording in 2 tabs using the inline recorder on Chrome on Android. The message is also shown to the user.

TTA

TTA (time to action) represents the time (in milliseconds) between an initial getUserMedia() attempt and its result, successful or otherwise. This value helps you complete the picture of what happened:

  • when the user has previously denied access you'll see values under 20ms
  • when the user has previously granted access this value measures how much time it takes to initialize the recording devices
  • when the user is prompted with the permissions dialog this value measures how much time it takes the user to react

If from the Date value you subtract TTA value you get the moment when getUserMedia() was executed.

Now that we've got a better grasp of the available data and its meaning, it's time to analyze it and see how we can extract meaningful analytics and relevant conclusions about different problems.

Analytics

Accurate permissions data on Chrome allow us to calculate 3 stats for your environment:

  • The percentage of times the permission dialog is shown
  • The percentage of times it is shown and dismissed
  • The percentage of times it is shown and the user blocked access

You can find these stats shown for each environment at the bottom of the Logs page.

There's also an Export all data to CSV option, which allows you export all the data to CSV format and import it in Excel or Google Sheets to make your own analysis.

Interpreting the data

You can draw a lot of conclusions by looking at the data in the table. To get you started let's go in depth with the following log entries:

Reading from the bottom to the top (green > orange > blue) we can find out what happened when a user tried to record with the HTML5 recorder:

  1. Green row: The user opened the recorder page in Chrome and was asked to allow the use of his camera and microphone (Cam and Mic columns both have the prompt value). Probably not realizing what it's about, the user ends up dismissing the permissions dialog, which results in a NotAllowedError with a Permission dismissed description. Between the time the browser prompted the user to allow the use of the camera and microphone and the time the user dismissed the permissions dialog a total of 3492ms passed. The value is usual for a human interaction - the act of dismissing the permissions dialog.
  2. Orange row: By looking at the IP we can conclude it is the same user. It seems the user refreshed the page (only 2 seconds passed between the 2 log entries), but because the user is on Chrome where the permissions are persistent and because the user previously dismissed the permissions dialog (Cam and Mic columns both have the denied value), Chrome automatically jumps to a NotAllowedError with a Permission denied description. The TTA value is very low, just 9ms, typical of an automatic response from Chrome.
  3. Blue row: the user tries a 3rd time after resettin the camera and microphone permissions from Chrome settings (Cam and Mic values have been reset to the prompt value). This time the user grants access when prompted, resulting in a OK status. He is also rather quick about it - it took him just 1123ms - but clearly a human interaction.

This are just a few examples of a number of possible scenarios, but they clearly demonstrate the usefulness of the logs when it comes to identifying a specific user issue.

Future developments

We plan on adding more useful stats in the page to help interpreting the data, but we are also looking at gathering data about connection problems. Stay tuned!