Recently, we learned that a domestic technical team has successfully developed an audio generation tool similar to "Pika" and has been testing it internally for six months. It is mainly used to add sound effects or background music to mute scenes in video production. Excitingly, this mysterious application is about to unveil its mystery and officially meet the public.
The technical team is called "SkyMapper" and they have collaborated with Huawei Cloud to launch a small tool called "Sora Opera". It is revealed that this tool will be officially launched within the next three months, giving users the opportunity to experience its powerful features.
SkyMapper has also provided a trial website for the Sora Opera project to Vision Future:
http://soraopera.skymaper.com, which will be open to the public in a few days.
It is worth mentioning that Sora Opera was first released on the Huawei Cloud platform and will later introduce an open-source version that can run on local machines to meet the needs of different users. The use of Sora Opera for video dubbing in the cloud is completely free, which undoubtedly provides great convenience for users.
Sora Opera has two core functions: sound effects for video production and video background music. Although the cloud version is initially available to partners due to computational cost, we believe that more users will be able to enjoy the surprises brought by these two functions in the near future.
The sound effects function allows users to input prompts such as "sound of sword hitting" or "pleasant bird chirping", and Sora Opera can quickly generate corresponding audio clips to add vivid sound effects to the video. The video background music function can generate background music with specific melodies and rhythms according to the user's needs, making the video more engaging.
In terms of efficiency, Sora Opera also performs well. According to SkyMapper, it only takes about 10 seconds to generate four 14-second videos with sound effects on a 1080ti graphics card. Users hardly feel any waiting time when using a higher-level 3090 graphics card. In addition, the professional version of Sora Opera supports full video dubbing for up to 60 minutes, allowing users to import their own footage and obtain new sound effects or inspiration.
In addition to generating audio for AI-generated videos, Sora Opera can also re-dub existing videos. It has powerful processing capabilities and can handle various complex visual contents. For example, it can accurately match sound effects even for accelerated videos. Sora Opera can also recognize the rhythm of the image and the relationship between sound and image outside the frame, such as the sound of clothes rubbing, making the dubbing effect more realistic and natural.
In terms of sound generation, Sora Opera also demonstrates amazing capabilities. It can understand the sound production rules of the real world and reproduce these sounds as accurately as possible. For example, when dubbing a video of a coffee machine making coffee, Sora Opera can capture the sound of the machine starting and stopping and perfectly match it with the process of coffee extraction and pouring shown in the video.
It is worth mentioning that Sora Opera also has the ability to re-dub damaged AI videos, which can produce magical effects. In addition, it has been specifically trained for green screen production materials and can accurately dub based on the characters' states, actions, and props in the frame. It also considers the possible sound sources and additional synchronous sound source data, providing more possibilities for video dubbing.
Although there are already various video production tools on the market, most of the available versions have mute clips, and the problem of adding background music to videos is still to be solved. However, with the continuous deepening of research in the AI music field, we believe that more excellent music generators will emerge in the future.
SkyMapper, as a company with a wide range of businesses, Sora Opera is just a by-product of their interactive film and game development process. However, this by-product demonstrates strong technical capabilities and market potential. At the end of last year, SkyMapper completed the production of a product called "Super Sensory Film Game", which uses AI to participate in the production of scripts, original drawings, animations, and music, greatly improving production efficiency. Currently, the production cycle of a single script has been shortened to 19 days.
This multiplayer interactive product uses AI to generate multi-line branching scripts, allowing each participant's choices to lead to different plot branches. Compared with existing products such as Werewolf and Script Kill, Super Sensory Film Game has a more stable experience and does not rely on a human host. Sora Opera provides professional production support for Super Sensory Film Game, providing powerful support for sound effects and background music.
Currently, SkyMapper has produced multiple Super Sensory Film Game works through Sora Opera, such as "Sea of Destiny," "Ghost Fortress," "Curse of the Sun God," "Dust," "Protagonist Academy," and more. These works not only demonstrate the powerful features of Sora Opera but also showcase SkyMapper's profound strength in the field of interactive film and games.
Sora Opera, as an emerging audio generation tool, demonstrates powerful capabilities in video production and background music. With the continuous improvement and optimization of its features, we believe it will bring more innovation and surprises to the field of video production in the future. At the same time, we also look forward to SkyMapper continuing to leverage its advantages in the interactive film and game field to bring us more high-quality works.
