What is Aeneas?

Aeneas is an advanced tool designed to automate the synchronization of audio and text, a process known as forced alignment. Developed using Python and C, Aeneas serves as a library and a collection of tools that facilitate the generation of synchronization maps between spoken audio and corresponding text fragments. This capability is particularly useful in various fields such as digital publishing, research, and multimedia content creation, where precise alignment of audio narration with text is essential.

The tool was developed by ReadBeyond, with Alberto Pettarin as the lead developer. It is released under the GNU Affero General Public License Version 3 (AGPL v3), making it open-source and freely available for use and modification. The latest version, 1.7.3, was released on March 15, 2017.

Features

Aeneas boasts a wide array of features that cater to the needs of users requiring audio-text synchronization. Below are some of the key features:

Synchronization Capabilities

• Forced Alignment: Aeneas can automatically compute the time intervals in audio files for each fragment of text, providing a synchronization map that indicates when each part of the text is spoken.

• Multilevel Alignment: The tool supports recursive alignment from paragraphs to sentences and down to word level, allowing for granular control over how audio and text are matched.

Input and Output Formats

• Supported Input Formats: Aeneas can process text files in various formats, including:

  • Parsed and plain text
  • Subtitles
  • Unparsed XML formats
  • Multilevel input text files

• Input Audio Formats: It can handle any audio file format that is readable by FFmpeg, making it highly versatile.

• Output Formats: The synchronization maps can be exported in multiple formats suitable for different applications:

  • Research: AUD, ELAN (EAF), TextGrid
  • Digital Publishing: SMIL for EPUB 3
  • Closed Captioning: SRT, SBV/SUB, TTML, WebVTT
  • Web: JSON
  • Further Processing: CSV, SSV, TSV, TXT, XML

Language Support

Aeneas is confirmed to work with 38 different languages, including but not limited to:

• English (ENG)
• Spanish (SPA)
• French (FRA)
• German (DEU)
• Japanese (JPN)

This extensive language support makes it suitable for a global audience.

TTS Integration

Aeneas includes several built-in Text-to-Speech (TTS) engine wrappers, such as:

• AWS Polly TTS API
• eSpeak (default)
• Festival
• MacOS 'say' command
• Nuance TTS API

This allows users to generate audio from text using various TTS engines, enhancing the tool's capabilities.

Batch Processing

Users can process multiple audio/text pairs in one go by creating a job container. This feature is particularly useful for large projects that require the synchronization of numerous files.

Robustness

Aeneas is designed to be robust against:

• Misspelled or mispronounced words
• Local rearrangements of words
• Background noise and sporadic audio spikes

This ensures that the synchronization remains accurate even in less-than-ideal audio conditions.

Fine-Tuning Options

The tool allows users to adjust splitting times and provides options for fine-tuning synchronization maps manually through an HTML output file. This is particularly useful for projects that demand high precision.

Extensive Testing

Aeneas comes with an extensive test suite, including over 1,200 unit, integration, and performance tests that must pass before each release. This commitment to quality ensures that users receive a reliable tool.

Use Cases

Aeneas can be applied in various scenarios, making it a versatile tool for different industries. Here are some common use cases:

Digital Publishing

• Audiobooks: Authors and publishers can use Aeneas to synchronize audio narrations with text in audiobooks, enhancing the reader’s experience.
• E-books: For e-books, particularly those in EPUB format, Aeneas can generate SMIL files to ensure that audio is correctly aligned with the text.

Education

• Language Learning: Educators can utilize Aeneas to create synchronized audio resources for language learners, helping them improve their listening and reading skills simultaneously.
• Interactive Learning Materials: Aeneas can be employed to develop interactive educational materials where audio and text are closely aligned, making learning more engaging.

Research

• Linguistic Studies: Researchers in linguistics can use Aeneas to analyze audio recordings and their corresponding transcripts, allowing for detailed studies of speech patterns and language use.
• Accessibility Research: Aeneas can be valuable for studies focused on accessibility in media, ensuring that closed captions and audio descriptions are accurately aligned with the content.

Multimedia Production

• Video Production: In video editing, Aeneas can assist in synchronizing voiceovers with on-screen text or subtitles, streamlining the post-production process.
• Podcasting: Podcasters can use Aeneas to create transcripts that are aligned with the audio, making it easier for listeners to follow along.

Pricing

Aeneas is an open-source tool, meaning it is available for free under the GNU Affero General Public License Version 3 (AGPL v3). Users can download, modify, and distribute the software without incurring any costs. However, users are encouraged to support the development of Aeneas through sponsorships, which can help improve the tool and add new features.

Comparison with Other Tools

When comparing Aeneas with other forced alignment tools available in the market, several unique selling points emerge:

Open Source vs. Proprietary

• Aeneas is open-source, allowing users to access the source code, modify it, and contribute to its development. In contrast, many other tools are proprietary, requiring users to purchase licenses and often lacking transparency.

Versatility in Formats

• Aeneas supports a broader range of input and output formats compared to some competitors. This versatility makes it suitable for various applications, from research to multimedia production.

Language Support

• With confirmed support for 38 languages, Aeneas stands out in its ability to cater to a diverse audience, unlike some tools that may only support a limited number of languages.

Robustness and Accuracy

• Aeneas is designed to handle background noise, mispronunciations, and other challenges that can affect audio quality. This robustness can lead to more accurate synchronization compared to other tools that may struggle under similar conditions.

Batch Processing

• The ability to batch process multiple audio/text pairs is a significant advantage for users working on large projects, as it can save time and streamline workflows.

FAQ

What are the system requirements for Aeneas?

Aeneas requires Python (preferably version 2.7.x), FFmpeg, and eSpeak to be installed on your system. It is compatible with Mac OS X, Windows, and deb-based Linux distributions.

How do I install Aeneas?

Installation can be done using an all-in-one installer for Mac OS X and Windows, or via a Bash script for Linux. Users can also download a VirtualBox+Vagrant virtual machine. The generic procedure involves installing Python, FFmpeg, and eSpeak, followed by using pip to install numpy and Aeneas.

Can Aeneas work with non-English languages?

Yes, Aeneas supports 38 languages, making it suitable for users working with various languages around the world.

Is there a user manual or documentation available?

Yes, Aeneas comes with extensive documentation that includes tutorials for both command-line tools and library usage. Users can refer to these resources for guidance on installation, usage, and troubleshooting.

How can I support the development of Aeneas?

Users interested in supporting Aeneas can reach out to the developers for sponsorship opportunities. Contributions can help improve the tool, fix bugs, and add new features.

Can I use Aeneas for commercial purposes?

Yes, since Aeneas is open-source and released under the AGPL v3 license, it can be used for commercial purposes as long as users comply with the terms of the license.

In conclusion, Aeneas is a powerful tool for anyone needing to synchronize audio and text efficiently and accurately. With its extensive features, versatility, and open-source nature, it stands out as a valuable asset for digital publishing, education, research, and multimedia production.