Screen copy android studio

How to programmatically capture screen on Android: a comprehensive guide

At Bolt, we encourage employees to use the products we’re building, to provide feedback that helps us continuously improve. To make this process as simple and efficient as possible, we’ve added a special button to the version of the Android app used by employees internally. Upon tap, it gathers device logs, captures screen image, prompts for a description and sends everything directly to special channel with support representatives. Building a robust screen capturing mechanisms has its pitfalls, and we want to share results and the knowledge we gained with the community.

In this article, we’re going to expand on different ways to programmatically capture a screenshot of your Android app. To explain the pros and cons of every approach, we’ll also briefly touch on various UI framework components such as WindowManager, Window, Surface, etc. and their role in screen rendering.

Even though this might not sound like a task you’d commonly be faced with at work, reading the article still might be beneficial, because having knowledge of the APIs we’re going to discuss here might come in handy for developing other features and will deepen your understanding of the Android UI framework.

For those who’d like to skip reading and jump right into the code, at Bolt we developed and open-sourced a library that combines all the discussed solutions and provides an easy-to-use API abstracted from the Android framework and the complexities of the underlying mechanisms:

bolteu/screenshotty

The library combines MediaProjection, PixelCopy and Canvas drawing and provides an easy-to-use API, abstracted from the…

To find out how it works internally, check out the screenshotty-lib module. The repo also contains documentation and a sample app. We hope that the community will find our work useful, and of course, anyone is welcome to contribute.

Starting from the basics

When we talk about capturing a screenshot, basically what we mean is creating a Bitmap consisting of all displayed elements.
From this, we can reason that displayed elements are Views and we know how to draw them on a Canvas, so the solution to our problem is as simple as:

Right? Well, not really. things are often a bit more complex.

There might be several DecorViews

If there’s a Dialog or a PopupWindow on the screen, they simply won’t be captured. To understand why that’s the case and what we can do about it, let’s make a quick overview of components involved in the View rendering process.

In the ActivityThread, when an Activity is created and attached to the Application context, internally it creates a Window and a WindowManager and registers different window callbacks. You can think of a Window as a container for a View hierarchy and the WindowManager is essentially the entity responsible for managing various properties of this container such as transforms, orientation, lifecycle, focus events, etc. Every Activity and Dialog has its own WindowManager instance, but internally all these instances share a WindowManagerGlobal singleton where all the logic for talking to the system WindowManager resides.

When you call setContentView, a DecorView is created inside the corresponding Window object. This View contains a content container and is also capable of showing things like floating action mode menus on top of it. This DecorView is then added to a local WindowManager which internally just delegates to the WindowManagerGlobal. A reference to the given View is saved and a special ViewRootImpl object is created. This object implements ViewParent interface and acts as a bridge between the window’s view hierarchy and low-level graphics framework components. We’ll get back to this later because now we have everything we need to solve the problem of dialogs not being captured.

As we mentioned, WindowManagerGlobal is a singleton which stores references to all the added Views and their position on the display. So, if we know how to draw one DecorView, we can compose several of them together to produce the final image. There are no public APIs available to retrieve them, but we can use reflection for this.

The drawing part doesn’t change a lot. We use LayoutParams of the corresponding views to correctly position them on the screenshot and apply the background dim you see when a dialog is displayed.

Accessing implementation internals is a very unreliable approach: different API versions might require special handling and, worst of all, the implementation might be changed by device vendors. To safeguard from having everything completely broken, we recommend the following approach as an add-on: first, a publicly accessible decor view of the visible activity is drawn, then we try to get displayed dialogs and render them on top, returning the original bitmap in case of failure. This way the worst-case scenario is having screenshots without dialogs, not that bad as opposed to no screenshots at all.

Another thing to keep in mind here is that WindowManagerGlobal is a process-level singleton, and some of the views we get from it might belong to stopped activities, i.e. those not visible to a user. To render only the dialogs and popups that belong to the current activity, we have to check that their context is our current Activity.

Capturing a Surface

So, we now know how to deal with dialogs and popups, but what if there’s a SurfaceView in our view hierarchy? It might be there for some custom Open GL drawing, camera preview or navigation maps. If we just draw the decor view on canvas — we’ll get blank pixels instead of the SurfaceView content. To understand why this happens, let’s take a brief look at where a Canvas used by Views during a drawing phase comes from.

We can think of Canvas as a kind of controller that allows us to mutate a buffer of pixel colors that are then sent for rendering. The mechanism for transferring these buffers (or actually their handles, because copying would be too expensive) is called BufferQueue. The idea here is simple — the producer takes a buffer from the queue, modifies it and puts back, while the consumer dequeues buffers and performs processing. The class, acting as an interface for Producers, is called Surface. Every window is backed by one and WindowManager is viewed as a Producer, sending buffers to SurfaceFlinger which then interacts with HardwareComposer to correctly compose buffers from different producers.

We can even take control of the window’s surface by doing this in our Activity:

If you do this, you’ll see that view hierarchy is not rendered, though views are there and events are captured. Only one client at a time can modify a buffer — a call to lockCanvas() or lockHardwareCanvas() acquires a buffer and creates a Canvas bound to it. When all modifications are done — unlockCanvasAndPost() sends the buffer back to the queue. Usually, all this is handled for us by ViewRootImpl class, which we’ve already mentioned before.

Now, what happens when there’s a SurfaceView in view hierarchy? For each such view, a new surface (i.e. a buffer queue) is requested from a SurfaceFlinger and the new surface is by default logically positioned behind the surface of the containing window. This means that the actual rendering of a SurfaceView content happens independently of View hierarchy rendering and two outputs are composed on a later stage of rendering. All such surfaces are scoped to the parent window.

The problem is that sitting on the producer side of the queue we can’t get access to buffers, nor do we have a generic hook for all SurfaceView implementations (Google Maps, for example, provides a snapshot() method that can be used to get the currently displayed content). Luckily, Android provides some public APIs that might suit our needs.

MediaProjection API (L+)

MediaProjection API was added in Android L and is based on VirtualDisplays, a mechanism that allows us to become buffer consumers. VirtualDisplay makes it possible to receive buffers composited by SurfaceFlinger to a Surface we provide during display creation.

To create a Surface we can use a SurfaceTexture or a class more convenient for this purpose — the ImageReader. ImageReader provides a callback to receive buffer contents wrapped into Image objects containing a raw ByteBuffer and meta-data about its contents.

When we create a VirtualDisplay using MediaProjection — it starts capturing all the user sees on their display (not only our app), and on onImageAvailable() callback fires every time a VSYNC for internal display is performed. Since we care only about one image — a flag indicating the first frame was consumed should be kept to avoid creating unnecessary bitmaps. The callback might be invoked multiple times even if we’ve closed the projection right after the first image was processed.

Even though MediaProjection is a solid solution for taking screenshots, it too has major downsides.

As it’s a very powerful mechanism that allows recording everything that’s happening on a display, that’s no surprise a user’s consent to initiate a capture session is required. When MediaProjection is requested, a system dialog asking for permission is displayed and we are able to create projection only after permission is granted. Pre-Android 10, the dialog had a “Don’t show again” option, but with the newest changes, the consent can’t be remembered. There’s a hack though: we can save the grant result for future uses, minimising the number of permission requests to one-per-process lifetime.

Also, the MediaProjection approach requires a lot of code to set up and has a very complicated flow with a lot of asynchronous methods and many possible points of failure. Screenshots produced by this method are as accurate as they get.

PixelCopy API (N+/O+)

PixelCopy is a not very well known API available for N+ devices that takes a Surface and a Bitmap as arguments and calls native code where the last queued buffer is peeked and rendered to a GL texture which is then copied to the Bitmap.

Starting from Android O, there’s an even more convenient method available: we can pass a window instead of a Surface and get all the surfaces scoped to its ViewRootImpl rendered on a Bitmap.

PixelCopy operates synchronously but this might change in the future, so to receive a callback you have to also provide a Handler to the method.

This approach is very simple:

It can be performed on any thread and the only requirement here is to ensure that view hierarchy has been drawn at least once, for which OnDrawListener can be used.

Even though it doesn’t capture dialogs and popups, it’s easy to combine it with the reflection-based approach we discussed above to produce quite accurate screenshots, without disrupting users as when using MediaProjection.

In conclusion

As you can see, programmatically capturing an accurate screenshot is a more complicated task than it may seem at first. There’s no right approach because every method discussed involves compromises and the choice has to be based on functional requirements.

Internally, screenshotty implements all these mechanisms, first trying to make a PixelCopy, falling back to MediaProjection and then to a canvas drawing if something fails. If this doesn’t match your needs, you can use our code as a reference — feel free to copy and modify it to achieve the desired behaviour.

We really hope you enjoyed this article and that it helps the developer community to do the things they do more easily!

And if you’re interested in working with us on Bolt app, check our Careers page.

Источник

Как записывать экран и делать скриншоты, используя Android API?

Android получил новый API на Kitkat и Lollipop, чтобы захватить видео на экране. Вы можете сделать это либо с помощью инструмента ADB, либо с помощью кода (начиная с Lollipop).

С тех пор, как новый API вышел, многие приложения пришли, которые используют эту функцию, позволяя записывать экран, и Microsoft даже сделал свой собственный Google-Now-On-tap конкурент приложение.

используя ADB, вы можете использовать:

вы даже можете сделать это из Android Studio себя.

проблема

Я не могу найти учебник или объяснение о том, как это сделать с помощью API, то есть в коде.

что я нашел

единственное место, которое я нашел, это документация (здесь, в разделе» захват экрана и совместное использование»), говоря мне это:

Android 5.0 позволяет добавлять захват экрана и общий доступ к экрану возможности для вашего приложения с новым android.сми.проекция Апис. Эта функция полезна, например, если вы хотите включить совместное использование экрана в приложении для видеоконференций.

новая createVirtualDisplay() метод позволяет вашему приложению захватывать содержимое главного экрана (дисплей по умолчанию) в поверхность объект, который ваше приложение может затем отправить по сети. Только API позволяет захватывать незащищенное содержимое экрана, а не системный звук. К начните захват экрана, ваше приложение должно сначала запросить пользователя разрешение на запуск диалог захвата экрана с использованием намерения получено через createScreenCaptureIntent() метод.

пример использования новых API см. В разделе MediaProjectionDemo класс в образце проекта.

дело в том, что я не могу найти образец «MediaProjectionDemo». Вместо этого я нашел образец «Screen Capture», но я не понимаю, как он работает, так как, когда я его запустил, все, что я видел, это мигающий экран, и я не думаю, что он сохраняет видео в файл. Образец кажется очень детская коляска.

вопросы

Как выполнить эти действия с помощью нового API:

1. начните запись, опционально включая аудио (микрофон/динамик/оба).
2. остановить запись
3. сделайте скриншот, а не видео.

кроме того, как настроить его (разрешение, запрошенный fps, цвета, время. ) ?

Источник

Screen copy android studio

This application provides display and control of Android devices connected via USB (or over TCP/IP). It does not require any root access. It works on GNU/Linux, Windows and macOS.

• lightness: native, displays only the device screen
• performance: 30

120fps, depending on the device

quality: 1920×1080 or above

low latency: 35

70ms
low startup time:

1 second to display the first image

non-intrusiveness: nothing is left installed on the device

user benefits: no account, no ads, no internet required

freedom: free and open source software

Its features include:

The Android device requires at least API 21 (Android 5.0).

Make sure you enabled adb debugging on your device(s).

On some devices, you also need to enable an additional option to control it using keyboard and mouse.

• Linux: apt install scrcpy
• Windows: download
• macOS: brew install scrcpy

On Debian and Ubuntu:

A Snap package is available: scrcpy .

For Fedora, a COPR package is available: scrcpy .

For Gentoo, an Ebuild is available: scrcpy/ .

For Windows, for simplicity, a prebuilt archive with all the dependencies (including adb ) is available:

It is also available in Chocolatey:

The application is available in Homebrew. Just install it:

You need adb , accessible from your PATH . If you don’t have it yet:

It’s also available in MacPorts, which sets up adb for you:

Plug an Android device, and execute:

It accepts command-line arguments, listed by:

Sometimes, it is useful to mirror an Android device at a lower definition to increase performance.

To limit both the width and height to some value (e.g. 1024):

The other dimension is computed to that the device aspect ratio is preserved. That way, a device in 1920×1080 will be mirrored at 1024×576.

The default bit-rate is 8 Mbps. To change the video bitrate (e.g. to 2 Mbps):

Limit frame rate

The capture frame rate can be limited:

This is officially supported since Android 10, but may work on earlier versions.

The device screen may be cropped to mirror only part of the screen.

This is useful for example to mirror only one eye of the Oculus Go:

If —max-size is also specified, resizing is applied after cropping.

Lock video orientation

To lock the orientation of the mirroring:

This affects recording orientation.

Some devices have more than one encoder, and some of them may cause issues or crash. It is possible to select a different encoder:

To list the available encoders, you could pass an invalid encoder name, the error will give the available encoders:

It is possible to record the screen while mirroring:

To disable mirroring while recording:

«Skipped frames» are recorded, even if they are not displayed in real time (for performance reasons). Frames are timestamped on the device, so packet delay variation does not impact the recorded file.

On Linux, it is possible to send the video stream to a v4l2 loopback device, so that the Android device can be opened like a webcam by any v4l2-capable tool.

The module v4l2loopback must be installed:

To create a v4l2 device:

This will create a new video device in /dev/videoN , where N is an integer (more options are available to create several devices or devices with specific IDs).

To list the enabled devices:

To start scrcpy using a v4l2 sink:

(replace N by the device ID, check with ls /dev/video* )

Once enabled, you can open your video stream with a v4l2-capable tool:

For example, you could capture the video within OBS.

It is possible to add buffering. This increases latency but reduces jitter (see #2464).

The option is available for display buffering:

Scrcpy uses adb to communicate with the device, and adb can connect to a device over TCP/IP. The device must be connected on the same network as the computer.

An option —tcpip allows to configure the connection automatically. There are two variants.

If the device (accessible at 192.168.1.1 in this example) already listens on a port (typically 5555) for incoming adb connections, then run:

If adb TCP/IP mode is disabled on the device (or if you don’t know the IP address), connect the device over USB, then run:

It will automatically find the device IP address, enable TCP/IP mode, then connect to the device before starting.

Alternatively, it is possible to enable the TCP/IP connection manually using adb :

Connect the device to the same Wi-Fi as your computer.

Get your device IP address, in Settings → About phone → Status, or by executing this command:

Enable adb over TCP/IP on your device: adb tcpip 5555 .

Unplug your device.

Connect to your device: adb connect DEVICE_IP:5555 (replace DEVICE_IP ).

Run scrcpy as usual.

It may be useful to decrease the bit-rate and the definition:

If several devices are listed in adb devices , you must specify the serial:

If the device is connected over TCP/IP:

You can start several instances of scrcpy for several devices.

Autostart on device connection

To connect to a remote device, it is possible to connect a local adb client to a remote adb server (provided they use the same version of the adb protocol).

Remote ADB server

To connect to a remote ADB server, make the server listen on all interfaces:

Warning: all communications between clients and ADB server are unencrypted.

Suppose that this server is accessible at 192.168.1.2. Then, from another terminal, run scrcpy:

By default, scrcpy uses the local port used for adb forward tunnel establishment (typically 27183 , see —port ). It is also possible to force a different tunnel port (it may be useful in more complex situations, when more redirections are involved):

To communicate with a remote ADB server securely, it is preferable to use a SSH tunnel.

First, make sure the ADB server is running on the remote computer:

Then, establish a SSH tunnel:

From another terminal, run scrcpy:

To avoid enabling remote port forwarding, you could force a forward connection instead (notice the -L instead of -R ):

From another terminal, run scrcpy:

Like for wireless connections, it may be useful to reduce quality:

By default, the window title is the device model. It can be changed:

Position and size

The initial window position and size may be specified:

To disable window decorations:

To keep the scrcpy window always on top:

The app may be started directly in fullscreen:

Fullscreen can then be toggled dynamically with MOD + f .

The window may be rotated:

Possibles values are:

• 0 : no rotation
• 1 : 90 degrees counterclockwise
• 2 : 180 degrees
• 3 : 90 degrees clockwise

The rotation can also be changed dynamically with MOD + ← (left) and MOD + → (right).

Note that scrcpy manages 3 different rotations:

• MOD + r requests the device to switch between portrait and landscape (the current running app may refuse, if it does not support the requested orientation).
• —lock-video-orientation changes the mirroring orientation (the orientation of the video sent from the device to the computer). This affects the recording.
• —rotation (or MOD + ← / MOD + → ) rotates only the window content. This affects only the display, not the recording.

Other mirroring options

To disable controls (everything which can interact with the device: input keys, mouse events, drag&drop files):

If several displays are available, it is possible to select the display to mirror:

The list of display ids can be retrieved by:

The secondary display may only be controlled if the device runs at least Android 10 (otherwise it is mirrored in read-only).

To prevent the device to sleep after some delay when the device is plugged in:

The initial state is restored when scrcpy is closed.

Turn screen off

It is possible to turn the device screen off while mirroring on start with a command-line option:

Or by pressing MOD + o at any time.

To turn it back on, press MOD + Shift + o .

On Android, the POWER button always turns the screen on. For convenience, if POWER is sent via scrcpy (via right-click or MOD + p ), it will force to turn the screen off after a small delay (on a best effort basis). The physical POWER button will still cause the screen to be turned on.

It can also be useful to prevent the device from sleeping:

Power off on close

To turn the device screen off when closing scrcpy:

For presentations, it may be useful to show physical touches (on the physical device).

Android provides this feature in Developers options.

Scrcpy provides an option to enable this feature on start and restore the initial value on exit:

Note that it only shows physical touches (with the finger on the device).

By default, scrcpy does not prevent the screensaver to run on the computer.

Rotate device screen

Press MOD + r to switch between portrait and landscape modes.

Note that it rotates only if the application in foreground supports the requested orientation.

Any time the Android clipboard changes, it is automatically synchronized to the computer clipboard.

Any Ctrl shortcut is forwarded to the device. In particular:

• Ctrl + c typically copies
• Ctrl + x typically cuts
• Ctrl + v typically pastes (after computer-to-device clipboard synchronization)

This typically works as you expect.

The actual behavior depends on the active application though. For example, Termux sends SIGINT on Ctrl + c instead, and K-9 Mail composes a new message.

To copy, cut and paste in such cases (but only supported on Android >= 7):

• MOD + c injects COPY
• MOD + x injects CUT
• MOD + v injects PASTE (after computer-to-device clipboard synchronization)

In addition, MOD + Shift + v allows to inject the computer clipboard text as a sequence of key events. This is useful when the component does not accept text pasting (for example in Termux), but it can break non-ASCII content.

WARNING: Pasting the computer clipboard to the device (either via Ctrl + v or MOD + v ) copies the content into the device clipboard. As a consequence, any Android application could read its content. You should avoid to paste sensitive content (like passwords) that way.

Some devices do not behave as expected when setting the device clipboard programmatically. An option —legacy-paste is provided to change the behavior of Ctrl + v and MOD + v so that they also inject the computer clipboard text as a sequence of key events (the same way as MOD + Shift + v ).

To disable automatic clipboard synchronization, use —no-clipboard-autosync .

To simulate «pinch-to-zoom»: Ctrl +click-and-move.

More precisely, hold Ctrl while pressing the left-click button. Until the left-click button is released, all mouse movements scale and rotate the content (if supported by the app) relative to the center of the screen.

Concretely, scrcpy generates additional touch events from a «virtual finger» at a location inverted through the center of the screen.

Physical keyboard simulation (HID)

By default, scrcpy uses Android key or text injection: it works everywhere, but is limited to ASCII.

On Linux, scrcpy can simulate a physical USB keyboard on Android to provide a better input experience (using USB HID over AOAv2): the virtual keyboard is disabled and it works for all characters and IME.

However, it only works if the device is connected by USB, and is currently only supported on Linux.

To enable this mode:

If it fails for some reason (for example because the device is not connected via USB), it automatically fallbacks to the default mode (with a log in the console). This allows to use the same command line options when connected over USB and TCP/IP.

In this mode, raw key events (scancodes) are sent to the device, independently of the host key mapping. Therefore, if your keyboard layout does not match, it must be configured on the Android device, in Settings → System → Languages and input → Physical keyboard.

This settings page can be started directly:

However, the option is only available when the HID keyboard is enabled (or when a physical keyboard is connected).

Text injection preference

There are two kinds of events generated when typing text:

• key events, signaling that a key is pressed or released;
• text events, signaling that a text has been entered.

By default, letters are injected using key events, so that the keyboard behaves as expected in games (typically for WASD keys).

But this may cause issues. If you encounter such a problem, you can avoid it by:

(but this will break keyboard behavior in games)

On the contrary, you could force to always inject raw key events:

These options have no effect on HID keyboard (all key events are sent as scancodes in this mode).

By default, holding a key down generates repeated key events. This can cause performance problems in some games, where these events are useless anyway.

To avoid forwarding repeated key events:

This option has no effect on HID keyboard (key repeat is handled by Android directly in this mode).

Right-click and middle-click

By default, right-click triggers BACK (or POWER on) and middle-click triggers HOME. To disable these shortcuts and forward the clicks to the device instead:

To install an APK, drag & drop an APK file (ending with .apk ) to the scrcpy window.

There is no visual feedback, a log is printed to the console.

Push file to device

To push a file to /sdcard/Download/ on the device, drag & drop a (non-APK) file to the scrcpy window.

There is no visual feedback, a log is printed to the console.

The target directory can be changed on start:

Audio is not forwarded by scrcpy. Use sndcpy.

In the following list, MOD is the shortcut modifier. By default, it’s (left) Alt or (left) Super .

It can be changed using —shortcut-mod . Possible keys are lctrl , rctrl , lalt , ralt , lsuper and rsuper . For example:

Super is typically the Windows or Cmd key.

Action	Shortcut
Switch fullscreen mode	MOD + f
Rotate display left	MOD + ← (left)
Rotate display right	MOD + → (right)
Resize window to 1:1 (pixel-perfect)	MOD + g
Resize window to remove black borders	MOD + w | Double-left-click¹
Click on HOME	MOD + h | Middle-click
Click on BACK	MOD + b | Right-click²
Click on APP_SWITCH	MOD + s | 4th-click³
Click on MENU (unlock screen)	MOD + m
Click on VOLUME_UP	MOD + ↑ (up)
Click on VOLUME_DOWN	MOD + ↓ (down)
Click on POWER	MOD + p
Power on	Right-click²
Turn device screen off (keep mirroring)	MOD + o
Turn device screen on	MOD + Shift + o
Rotate device screen	MOD + r
Expand notification panel	MOD + n | 5th-click³
Expand settings panel	MOD + n + n | Double-5th-click³
Collapse panels	MOD + Shift + n
Copy to clipboard⁴	MOD + c
Cut to clipboard⁴	MOD + x
Synchronize clipboards and paste⁴	MOD + v
Inject computer clipboard text	MOD + Shift + v
Enable/disable FPS counter (on stdout)	MOD + i
Pinch-to-zoom	Ctrl +click-and-move
Drag & drop APK file	Install APK from computer
Drag & drop non-APK file	Push file to device

¹Double-click on black borders to remove them.
²Right-click turns the screen on if it was off, presses BACK otherwise.
³4th and 5th mouse buttons, if your mouse has them.
⁴Only on Android >= 7.

Shortcuts with repeated keys are executted by releasing and pressing the key a second time. For example, to execute «Expand settings panel»:

1. Press and keep pressing MOD .
2. Then double-press n .
3. Finally, release MOD .

All Ctrl +key shortcuts are forwarded to the device, so they are handled by the active application.

To use a specific adb binary, configure its path in the environment variable ADB :

To override the path of the scrcpy-server file, configure its path in SCRCPY_SERVER_PATH .

To override the icon, configure its path in SCRCPY_ICON_PATH .

A colleague challenged me to find a name as unpronounceable as gnirehtet.

strcpy copies a string; scrcpy copies a screen.

Источник