Android media source code

MediaCodec или понимаем как хотим

С выходом Android 4.3 (API 18), Google привнесла долгожданный компонент под названием MediaCodec. Класс был открыт публике с выходом API 16, но для нормального использования и поддержки в Android системе требуется минимальный уровень API 18.

Материал рассчитан на опытного Android разработчика. Я попробую объяснить и показать примеры кодирования видео на лету с использованием Surface как входа и выхода потока данных. Если интересно, прошу под кат.

Что такое MediaCodec?

MediaCodec class can be used to access low-level media codec, i.e. encoder/decoder components.

MediaCodec класс может быть использован для доступа к низко-уровневому медиа кодеку, т.е. кодеру/декодеру

В принципе это кодер или декодер который манипулирует буферами данных. Если мы будем смотреть на формат H264, он же video/avc, то по сути буфер будет хранить NAL кадры и т.д.

Зачем нужен MediaCodec?

В большинстве случаев Android разработчики будут использовать VideoView, MediaPlayer с SurfaceView и этого вполне достаточно. Но как только речь зайдет о создании видео потока для дальнейшей передачи куда-либо, у нас не так много вариантов. Тут нам поможет MediaCodec.

Как кодировать?

MediaCodec дает возможность создать Surface объект для принятия данных для кодера. Я разделил логику кодирвоания на три этапа: подготовка, цикл кодирования, освобождение ресурсов.

Так как процесс кодирования по сути простой цикл обработки буферов данных, нам потребуется создать отдельный поток, который будет представлять непосредственно видео кодер.

Подготовка

Прежде чем приступить к кодированию нам потребуется описать формат и конфигурацию видео на выходе, создать сам кодер и получить Surface объект для ввода данных кодеру.

Цикл кодирования

mBufferInfo и data хранят данные о кадре(ах) который по сути может быть передан по сети или записаны в файл и др. Эти же данные могут быть декодированы h264 кодеком, чтобы получить изображение.

Важно заметить, что BufferInfo класс имеет поле flags. По сути может содержать три флага: BUFFER_FLAG_CODEC_CONFIG, BUFFER_FLAG_END_OF_STREAM и BUFFER_FLAG_SYNC_FRAME.

Давайте рассмотрим каждый из флагов:

• BUFFER_FLAG_CODEC_CONFIG флаг сигнализирует о том, что буфер данных содержит конфигурацию кодека. Первый доступный буфер кодера должен содержать данный флаг. К примеру для нашего H264 это может быть csd-0, который может быть использован во время декодирования при создании видео формата кодека.
• BUFFER_FLAG_END_OF_STREAM флаг сигнализирует о том, что достигнут конец потока данных. Я не уверен на счет использования данного флага во время кодирования, т.к. мы контролируем поток данных.
• BUFFER_FLAG_SYNC_FRAME флаг сигнализирует о том, что кадр является I-Frame. Важно использовать при передачи в файл или по сети, к примеру по протоколу RTP.

Освобождение ресурсов

Тут все просто, останавливаем кодер и освобождаем системные ресурсы.

Что дальше?

Имея Surface возможны как минимум три пути ввода данных: MediaPlayer (Camera?), OpenGL, Canvas. По примеру использования рендера в SurfaceView возможно создать подобный цикл на основе нашего Surface:

Вариантов использования может быть несколько. Еще интересный момент, Surface является Parcelable, что дает возможность передачи через Binder в другие процессы, таким образом ваш кодер может находиться в одном процессе, когда рендер в другом.

Как декодировать?

Процесс декодирования схожий на процесс кодирования. Мы так же манипулируем буферами данных. Данный процесс я разделил на четыре части: конфигурация, декодирование сэмпла, цикл декодирования, освобождение ресурсов.

Я опущу детали создания класса потока, архитектура схожа на ту, которую я использовал в процессе кодирования.

Конфигурация

Процесс конфигурации критичен, иначе кодек невозможно создать и использовать для декодирования поступающих данных.

Важно заметить, мы обязаны знать на какой Surface нужно выводить изображение, так же ширину и высоту изображения. csd-0 так же важен на этом этапе, как было описанно ранее в процессе кодирования, первый буфер данных должен иметь флаг BUFFER_FLAG_CODEC_CONFIG, этот буфер и является csd-0 который необходимо передать на этапе конфигурации декодера.

Декодирование сэмпла

Как только кодированные данные доступны их необходимо передавать декодеру. Мы запрашиваем буфер декодера, как только он доступен, передаем наши данные.

Если данные были переданы по сети, к примеру RTP протокол, необходимо учесть, что сэмплы передаются последовательно, иначе возможны артифакты при выводе изображения. На данном этапе, само изображение не готово для вывода на Surface, мы просто передаем известные данные декодеру.

Цикл декодирования

Цикл декодирования включает в себя запрос буфера данных который доступен на выходе, если доступен, можно его необходимо вернуть системе и сообщить если мы желаем рендерить данный кадр на Surface.

Источник

Music Player In Android Studio With Source Code

An Android music player made developed using Java in Android Studio that can access all media files within your phone and list the tracks for you. First running the application will prompt you to access your storage, be sure to click allow. The Android music player then lists all your media files of your device. You can also track based on Playlist, Artists, Genres with the menu on your top left. Music is also shown based on Albums. This software works just for your mobile default music player.

Screenshot

Feaures of Android Music Player software

• Play music with one-touch
• Scan all media files of your device
• See the source and folders of your media files
• Custom controls to play, pause, skip, next song, previous and fast-forwarding
• The navigation bar to browse media on basis of Albums, genres, artists…

Installation Steps:

To run the software, you will need to download Android Studio on your PC. First, download and then run Android Studio. You also need to install the Android SDK or Virtual Devices component to run within the visual studio or you can use your own android phone to install the application using a USB cable. You will also need to Unzip the .zip file using any zip programs such as Winrar or 7Zip. Download the zip file and unzip the .zip file using any zip programs such as Winrar or 7Zip. After extracting, copy the project folder to your designated location and open your folder or project using Android Studio.

If you need help setting up your software components, follow this video:

• Download and extract the .zip file
• Open the folder using Android Studio

FIND MORE on this site.

DOWNLOAD Music Player In Android Studio With Source Code FOR FREE

Источник

Android media source code

ExoPlayer

ExoPlayer is an application level media player for Android. It provides an alternative to Android’s MediaPlayer API for playing audio and video both locally and over the Internet. ExoPlayer supports features not currently supported by Android’s MediaPlayer API, including DASH and SmoothStreaming adaptive playbacks. Unlike the MediaPlayer API, ExoPlayer is easy to customize and extend, and can be updated through Play Store application updates.

• The developer guide provides a wealth of information.
• The class reference documents ExoPlayer classes.
• The release notes document the major changes in each release.
• Follow our developer blog to keep up to date with the latest ExoPlayer developments!

ExoPlayer modules can be obtained from the Google Maven repository. It’s also possible to clone the repository and depend on the modules locally.

From the Google Maven repository

1. Add ExoPlayer module dependencies

The easiest way to get started using ExoPlayer is to add it as a gradle dependency in the build.gradle file of your app module. The following will add a dependency to the full library:

where 2.X.X is your preferred version.

As an alternative to the full library, you can depend on only the library modules that you actually need. For example the following will add dependencies on the Core, DASH and UI library modules, as might be required for an app that only plays DASH content:

When depending on individual modules they must all be the same version.

The available library modules are listed below. Adding a dependency to the full ExoPlayer library is equivalent to adding dependencies on all of the library modules individually.

• exoplayer-core : Core functionality (required).
• exoplayer-dash : Support for DASH content.
• exoplayer-hls : Support for HLS content.
• exoplayer-rtsp : Support for RTSP content.
• exoplayer-smoothstreaming : Support for SmoothStreaming content.
• exoplayer-transformer : Media transformation functionality.
• exoplayer-ui : UI components and resources for use with ExoPlayer.

In addition to library modules, ExoPlayer has extension modules that depend on external libraries to provide additional functionality. Some extensions are available from the Maven repository, whereas others must be built manually. Browse the extensions directory and their individual READMEs for details.

More information on the library and extension modules that are available can be found on the Google Maven ExoPlayer page.

2. Turn on Java 8 support

If not enabled already, you also need to turn on Java 8 support in all build.gradle files depending on ExoPlayer, by adding the following to the android section:

3. Enable multidex

If your Gradle minSdkVersion is 20 or lower, you should enable multidex in order to prevent build errors.

Cloning the repository and depending on the modules locally is required when using some ExoPlayer extension modules. It’s also a suitable approach if you want to make local changes to ExoPlayer, or if you want to use a development branch.

First, clone the repository into a local directory and checkout the desired branch:

Next, add the following to your project’s settings.gradle file, replacing path/to/exoplayer with the path to your local copy:

You should now see the ExoPlayer modules appear as part of your project. You can depend on them as you would on any other local module, for example:

• Development work happens on the dev-v2 branch. Pull requests should normally be made to this branch.
• The release-v2 branch holds the most recent release.

Using Android Studio

To develop ExoPlayer using Android Studio, simply open the ExoPlayer project in the root directory of the repository.

Источник

Access media files from shared storage

To provide a more enriched user experience, many apps allow users to contribute and access media that’s available on an external storage volume. The framework provides an optimized index into media collections, called the media store, that allows for retrieving and updating these media files more easily. Even after your app is uninstalled, these files remain on the user’s device.

To interact with the media store abstraction, use a ContentResolver object that you retrieve from your app’s context:

Kotlin

The system automatically scans an external storage volume and adds media files to the following well-defined collections:

• Images, including photographs and screenshots, which are stored in the DCIM/ and Pictures/ directories. The system adds these files to the MediaStore.Images table.
• Videos, which are stored in the DCIM/ , Movies/ , and Pictures/ directories. The system adds these files to the MediaStore.Video table.
• Audio files, which are stored in the Alarms/ , Audiobooks/ , Music/ , Notifications/ , Podcasts/ , and Ringtones/ directories. Additionally, the system recognizes audio playlists that are in the Music/ or Movies/ directories, as well as voice recordings that are in the Recordings/ directory. The system adds these files to the MediaStore.Audio table. The recordings directory isn’t available on Android 11 (API level 30) and lower.
• Downloaded files, which are stored in the Download/ directory. On devices that run Android 10 (API level 29) and higher, these files are stored in the MediaStore.Downloads table. This table isn’t available on Android 9 (API level 28) and lower.

The media store also includes a collection called MediaStore.Files . Its contents depend on whether your app uses scoped storage, available on apps that target Android 10 or higher:

• If scoped storage is enabled, the collection shows only the photos, videos, and audio files that your app has created. Most developers won’t need to use MediaStore.Files to view media files from other apps, but if you have a specific requirement to do so, you can declare the READ_EXTERNAL_STORAGE permission. It’s recommended, however, that you use the MediaStore APIs to open files that your app hasn’t created.
• If scoped storage is unavailable or not being used, the collection shows all types of media files.

Request necessary permissions

Before performing operations on media files, make sure your app has declared the permissions that it needs to access these files. Keep in mind, however, that your app shouldn’t declare permissions that it doesn’t need or use.

Storage permission

The permissions model for accessing media files in your app depends on whether your app uses scoped storage, available on apps that target Android 10 or higher.

Scoped storage enabled

If your app uses scoped storage, it should request storage-related permissions only for devices that run Android 9 (API level 28) or lower. You can apply this condition by adding the android:maxSdkVersion attribute to the permission declaration in your app’s manifest file:

Don’t unnecessarily request storage-related permissions for devices that run Android 10 or higher. Your app can contribute to well-defined media collections, including the MediaStore.Downloads collection, without requesting any storage-related permissions. If you’re developing a camera app, for example, you don’t need to request storage-related permissions because your app owns the images that you’re writing to the media store.

To access files that other apps have created, the following conditions must each be true:

If your app wants to access a file within the MediaStore.Downloads collection that your app didn’t create, you must use the Storage Access Framework. To learn more about how to use this framework, see the guide on how to access documents and other files.

Scoped storage unavailable

If your app is used on a device that runs Android 9 or lower, or if your app has temporarily opted out of scoped storage, you must request the READ_EXTERNAL_STORAGE permission to access media files. If you want to modify media files, you must request the WRITE_EXTERNAL_STORAGE permission, as well.

Media location permission

If your app targets Android 10 (API level 29) or higher, in order for your app to retrieve unredacted Exif metadata from photos, you need to declare the ACCESS_MEDIA_LOCATION permission in your app’s manifest, then request this permission at runtime.

Check for updates to the media store

To access media files more reliably, particularly if your app caches URIs or data from the media store, check whether the media store version has changed compared to when you last synced your media data. To perform this check for updates, call getVersion() . The returned version is a unique string that changes whenever the media store changes substantially. If the returned version is different from the last synced version, rescan and resync your app’s media cache.

Complete this check at app process startup time. There’s no need to check the version each time you query the media store.

Don’t assume any implementation details regarding the version number.

Query a media collection

To find media that satisfies a particular set of conditions, such as a duration of 5 minutes or longer, use an SQL-like selection statement similar to the one shown in the following code snippet:

Kotlin

When performing such a query in your app, keep the following in mind:

• Call the query() method in a worker thread.
• Cache the column indices so that you don’t need to call getColumnIndexOrThrow() each time you process a row from the query result.
• Append the ID to the content URI, as shown in the code snippet.
• Devices that run Android 10 and higher require column names that are defined in the MediaStore API. If a dependent library within your app expects a column name that’s undefined in the API, such as «MimeType» , use CursorWrapper to dynamically translate the column name in your app’s process.

Load file thumbnails

If your app shows multiple media files and requests that the user choose one of these files, it’s more efficient to load preview versions—or thumbnails—of the files instead of the files themselves.

To load the thumbnail for a given media file, use loadThumbnail() and pass in the size of the thumbnail that you want to load, as shown in the following code snippet:

Kotlin

Open a media file

The specific logic that you use to open a media file depends on whether the media content is best represented as a file descriptor, a file stream, or a direct file path:

File descriptor

To open a media file using a file descriptor, use logic similar to that shown in the following code snippet:

Kotlin

File stream

To open a media file using a file stream, use logic similar to that shown in the following code snippet:

Kotlin

Direct file paths

To help your app work more smoothly with third-party media libraries, Android 11 (API level 30) and higher allow you to use APIs other than the MediaStore API to access media files from shared storage. You can instead access media files directly using either of the following APIs:

• The File API.
• Native libraries, such as fopen() .

If you don’t have any storage-related permissions, you can access files in your app-specific directory, as well as media files that are attributed to your app, using the File API.

If your app tries to access a file using the File API and it doesn’t have the necessary permissions, a FileNotFoundException occurs.

To access other files in shared storage on a device that runs Android 10 (API level 29), it’s recommended that you temporarily opt out of scoped storage by setting requestLegacyExternalStorage to true in your app’s manifest file. In order to access media files using native files methods on Android 10, you must also request the READ_EXTERNAL_STORAGE permission.

Considerations when accessing media content

When accessing media content, keep in mind the considerations discussed in the following sections.

Cached data

If your app caches URIs or data from the media store, periodically check for updates to the media store. This check allows your app-side, cached data to stay in sync with the system-side, provider data.

Performance

When you perform sequential reads of media files using direct file paths, the performance is comparable to that of the MediaStore API.

When you perform random reads and writes of media files using direct file paths, however, the process can be up to twice as slow. In these situations, we recommend using the MediaStore API instead.

DATA column

When you access an existing media file, you can use the value of the DATA column in your logic. That’s because this value has a valid file path. However, don’t assume that the file is always available. Be prepared to handle any file-based I/O errors that could occur.

To create or update a media file, on the other hand, don’t use the value of the DATA column. Instead, use the values of the DISPLAY_NAME and RELATIVE_PATH columns.

Storage volumes

Apps that target Android 10 or higher can access the unique name that the system assigns to each external storage volume. This naming system helps you efficiently organize and index content, and it gives you control over where new media files are stored.

The following volumes are particularly useful to keep in mind:

• The VOLUME_EXTERNAL volume provides a view of all shared storage volumes on the device. You can read the contents of this synthetic volume, but you cannot modify the contents.
• The VOLUME_EXTERNAL_PRIMARY volume represents the primary shared storage volume on the device. You can read and modify the contents of this volume.

You can discover other volumes by calling MediaStore.getExternalVolumeNames() :

Kotlin

Location where media was captured

Some photographs and videos contain location information in their metadata, which shows the place where a photograph was taken or where a video was recorded.

To access this location information in your app, use one API for photograph location information and another API for video location information.

Photographs

If your app uses scoped storage, the system hides location information by default. To access this information, complete the following steps:

Request the ACCESS_MEDIA_LOCATION permission in your app’s manifest.

From your MediaStore object, get the exact bytes of the photograph by calling setRequireOriginal() and pass in the URI of the photograph, as shown in the following code snippet:

Kotlin

Videos

To access location information within a video’s metadata, use the MediaMetadataRetriever class, as shown in the following code snippet. Your app doesn’t need to request any additional permissions to use this class.

Kotlin

Sharing

Some apps allow users to share media files with each other. For example, social media apps give users the ability to share photos and videos with friends.

To share media files, use a content:// URI, as recommended in the guide to creating a content provider.

App attribution of media files

When scoped storage is enabled for an app that targets Android 10 or higher, the system attributes an app to each media file, which determines the files that your app can access when it hasn’t requested any storage permissions. Each file can be attributed to only one app. Therefore, if your app creates a media file that’s stored in the photos, videos, or audio files media collection, your app has access to the file.

If the user uninstalls and reinstalls your app, however, you must request READ_EXTERNAL_STORAGE to access the files that your app originally created. This permission request is required because the system considers the file to be attributed to the previously-installed version of the app, rather than the newly-installed one.

Add an item

To add a media item to an existing collection, call code similar to the following. This code snippet accesses the VOLUME_EXTERNAL_PRIMARY volume on devices that run Android 10 or higher. That’s because, on these devices, you can only modify the contents of a volume if it’s the primary volume, as described in the storage volumes section.

Kotlin

Toggle pending status for media files

If your app performs potentially time-consuming operations, such as writing to media files, it’s useful to have exclusive access to the file as it’s being processed. On devices that run Android 10 or higher, your app can get this exclusive access by setting the value of the IS_PENDING flag to 1. Only your app can view the file until your app changes the value of IS_PENDING back to 0.

The following code snippet builds upon the previous code snippet. The following snippet shows how to use the IS_PENDING flag when storing a long song in the directory corresponding to the MediaStore.Audio collection:

Kotlin

Give a hint for file location

When your app stores media on a device running Android 10, the media is organized based on its type by default. For example, new image files are placed by default in the Environment.DIRECTORY_PICTURES directory, which corresponds to the MediaStore.Images collection.

If your app is aware of a specific location where files should be stored, such as a photo album called Pictures/MyVacationPictures, you can set MediaColumns.RELATIVE_PATH to provide the system a hint for where to store the newly-written files.

Update an item

To update a media file that your app owns, run code similar to the following:

Kotlin

If scoped storage is unavailable or not enabled, the process shown in the preceding code snippet also works for files that your app doesn’t own.

Update in native code

If you need to write media files using native libraries, pass the file’s associated file descriptor from your Java-based or Kotlin-based code into your native code.

The following code snippet shows how to pass a media object’s file descriptor into your app’s native code:

Kotlin

Update other apps’ media files

If your app uses scoped storage, it ordinarily cannot update a media file that a different app contributed to the media store.

It’s still possible to get user consent to modify the file, however, by catching the RecoverableSecurityException that the platform throws. You can then request that the user grant your app write access to that specific item, as shown in the following code snippet:

Kotlin

Complete this process each time your app needs to modify a media file that it didn’t create.

Alternatively, if your app runs on Android 11 or higher, you can allow users to grant your app write access to a group of media files. Call the createWriteRequest() method, as described in the section on how to manage groups of media files.

If your app has another use case that isn’t covered by scoped storage, file a feature request and temporarily opt out of scoped storage.

Remove an item

To remove an item that your app no longer needs in the media store, use logic similar to what’s shown in the following code snippet:

Kotlin

If scoped storage is unavailable or isn’t enabled, you can use the preceding code snippet to remove files that other apps own. If scoped storage is enabled, however, you need to catch a RecoverableSecurityException for each file that your app wants to remove, as described in the section on updating media items.

If your app runs on Android 11 or higher, you can allow users to choose a group of media files to remove. Call the createTrashRequest() method or the createDeleteRequest() method, as described in the section on how to manage groups of media files.

If your app has another use case that isn’t covered by scoped storage, file a feature request and temporarily opt out of scoped storage.

Detect updates to media files

Your app might need to identify storage volumes containing media files that apps added or modified, compared to a previous point in time. To detect these changes most reliably, pass the storage volume of interest into getGeneration() . As long as the media store version doesn’t change, the return value of this method monotonically increases over time.

In particular, getGeneration() is more robust than the dates in media columns, such as DATE_ADDED and DATE_MODIFIED . That’s because those media column values could change when an app calls setLastModified() , or when the user changes the system clock.

Manage groups of media files

On Android 11 and higher, you can ask the user to select a group of media files, then update these media files in a single operation. These methods offer better consistency across devices, and the methods make it easier for users to manage their media collections.

The methods that provide this «batch update» functionality include the following:

createWriteRequest() Request that the user grant your app write access to the specified group of media files. createFavoriteRequest() Request that the user marks the specified media files as some of their «favorite» media on the device. Any app that has read access to this file can see that the user has marked the file as a «favorite». createTrashRequest()

Request that the user place the specified media files in the device’s trash. Items in the trash are permanently deleted after a system-defined time period.

Request that the user permanently delete the specified media files immediately, without placing them in the trash beforehand.

After calling any of these methods, the system builds a PendingIntent object. After your app invokes this intent, users see a dialog that requests their consent for your app to update or delete the specified media files.

For example, here is how to structure a call to createWriteRequest() :

Kotlin

Evaluate the user’s response. If the user provided consent, proceed with the media operation. Otherwise, explain to the user why your app needs the permission:

Kotlin

Media management permission

Users might trust a particular app to perform media management, such as making frequent edits to media files. If your app targets Android 11 or higher and isn’t the device’s default gallery app, you must show a confirmation dialog to the user each time your app attempts to modify or delete a file.

If your app targets Android 12 (API level 31) or higher, you can request that users grant your app access to the Media management special permission. This permission allows your app to do each of the following without needing to prompt the user for each file operation:

To do so, complete the following steps:

Declare the MANAGE_MEDIA permission and the READ_EXTERNAL_STORAGE permission in your app’s manifest file.

In order to call createWriteRequest() without showing a confirmation dialog, declare the ACCESS_MEDIA_LOCATION permission as well.

In your app, show a UI to the user to explain why they might want to grant media management access to your app.

Invoke the ACTION_REQUEST_MANAGE_MEDIA intent action. This takes users to the Media management apps screen in system settings. From here, users can grant the special app access.

Use cases that require an alternative to media store

If your app primarily performs one of the following roles, consider an alternative to the MediaStore APIs.

Work with other types of files

If your app works with documents and files that don’t exclusively contain media content, such as files that use the EPUB or PDF file extension, use the ACTION_OPEN_DOCUMENT intent action, as described in the guide on how to store and access documents and other files.

File sharing in companion apps

In cases where you provide a suite of companion apps—such as a messaging app and a profile app—set up file sharing using content:// URIs. We also recommend this workflow as a security best practice.

Additional resources

For more information about how to store and access media, consult the following resources.

Samples

Videos

Content and code samples on this page are subject to the licenses described in the Content License. Java is a registered trademark of Oracle and/or its affiliates.

Источник