A: The MusicGen model within AudioCraft can, to some extent, achieve the task of converting a sung audio melody into an audio of musical instruments.
AudioCraft is a PyTorch library developed by Facebook Research. The MusicGen model has the ability to generate music based on text and melody conditions. It has been trained on a vast amount of music data and can create musical works with various instruments according to the input melody and other information. For example, users can utilize MusicGen through ways such as the Hugging Face Space web page, Google Colab, or running it locally. When using it, upload the reference audio containing the sung audio melody, and combine it with some text descriptions to assist in defining the style of the generated music, the types of instruments, etc. In theory, you can then obtain an audio played by musical instruments that is related to the original melody.