Save audio to google cloud storage via Firebase cloud function

Yiling Liu
4 min readSep 28, 2019

--

The problem is pretty straight forward, we want to create a button, you can click on it to record sound, then click save it’ll be uploaded to google cloud storage as a mp3 file.

codex

mux

demux

stream

buffer

thought it’s simple and was browsing through this page:

Hmm, looks pretty straight forward and I was following it happily, until I found it doesn’t make sense at all. There is this error message:

TypeError: admin.storage(...).bucket(...).ref is not a function

When I’m using

// Create a root reference
var storageRef = firebase.storage().ref();

A bit of dig into it and the reason is this:

Inside cloud function, it’s using Node.js 8 with Firebase Admin SDK. But the tutorial is referring to the Web API for Firebase. It is so confusing! They should at least put a page stating the difference between vague web section and Cloud function!

The right documents for cloud function are here: sample code, API reference.

Oh well.

So by browsing through the API reference I found there are 2 ways to save an audio file to GCS.

Solution 1: Create a Bucket class, then use upload method.

bucket.upload('/local/path/image.png', 
function(err, file, apiResponse) {}
);

The thing is the first parameter should be a local path. Witch means I need to first download it temporarily then upload it then delete the temporary file. As I’m not doing any processing or crazy modification(unlike this example) to the sound data, so it’s a waste of effort.

Solution 2: Create a File class, then use save method.

file.save(contents, function(err) {   
if (!err) {
// File written successfully.
}
});

I like this one as I can just use contents, which I’ll get from request.body.

My code:

exports.saveAudio = functions.https.onRequest(async (req, res) => {
const uuidv1 = require('uuid/v1');
console.log('Saving audio to GCS.');
var bucket = admin.storage().bucket('my-bucket-name');
const options = {
metadata: {
contentType: 'audio/mpeg',
}
};
var uuid = uuidv1();
var file = bucket.file(`audio/${uuid}.mp3`);
await file.save(req.body, options)
.then(stuff => {
console.log('Audio saved successfully.');
return file.getSignedUrl({
action: 'read',
expires: '03-09-2500'
})
})
.then(urls => {
var url = urls[0];
console.log(`Audio url = ${url}`)
res.status(200).send(JSON.stringify(url));
return url
})
.catch(err => {
console.log(`Unable to upload audio ${err}`)
});
});

So first we use uuidv1 to generate random and unique ids as audio link, then we upload audio to this audio link in GCS, and return a signedUrl.

I did got another error:

Unable to upload audio SigningError: Permission iam.serviceAccounts.signBlob is required to perform this operation on service account projects/my-project/serviceAccounts/my-project@appspot.gserviceaccount.com.

And it’s because currently my project is using the default service account, my-project@appspot.gserviceaccount.com but it’s only given Editor role, which doesn’t have permission iam.serviceAccounts.signBlob. The solution is to just go to your IAM and add another role called Cloud Functions Service Agent to it.

Transforming webm to mp3

webm is Container for VP8/VP9/AV1 video while Vorbis/Opus is for audio. We need to convert the webm to mp3 because iphone doesn’t support webm. Damn iphone.

I knew this woudn’t be simple, but I ddin’t know that it was THAT hard.

I found a tutorial here on github which didn’t not work at all, because it’s using this package:

const ffmpeg_static = require('ffmpeg-static');

I cannot install this package at all, there is some lock problem.

Luckily I found another example which uses these 3 lines in the beginning of the code to import.

const ffmpegPath = require('@ffmpeg-installer/ffmpeg').path;
const ffmpeg = require('fluent-ffmpeg');
ffmpeg.setFfmpegPath(ffmpegPath);

And then there is this promisifyCommand to tell us then would the conversion be finished.

// Makes an ffmpeg command return a promise.
function promisifyCommand(command) {
return new Promise((resolve, reject) => {
command.on('end', resolve).on('error', reject).run();
});
}

At last you use these 3 lines of code to convert webm to mp3.

var command = new ffmpeg('/tmp/sound.webm')
.toFormat('mp3')
.save('/tmp/sound.mp3');
await promisifyCommand(command);

Buffer buffer

Now we can successfully convert a good webm to mp3, we just need to know how to save a webm file to local temp location, then use the convert.

The way I was using to save to a webm file is like this:

const nodeBuffer = Buffer.from(req.body, 'base64');
fs.writeFileSync(tempLocalPath, nodeBuffer);

However, when I’m trying to use ffmpeg to convert my webm file, it’s always giving me this error:

An error occurred: ffmpeg exited with code 1: /tmp/da1ba0a0-f7d1-11e9-9a94-25e0ae7f6812.webm: Invalid data found when processing input

Also when I’m checking the webm file locally, it’s really small comparing with the original webm file.

So the way it works is req.body is an encoded string (you encode binary to a string), so now you tell the buffer the string and the way it was encoded(base64), you can get the real binary data, which is a webm file. So I’m checking how much my buffer is and it’s always showing a really small number(70).

console.log(nodeBuffer.byteLength);

The problem here, at last I found out, is the Content-Type, I didn’t set it to audio/webm, so it doesn’t know it’s communicating binary data, and that’s why it’s so small.

After I added Content-Type in the header, it worked!

--

--