I’ve recently been involved in a POC for a on-premise SharePoint 2013 installation where the customer wanted to make videos available in SharePoint but wanted them to be stored in Azure for storage/scalability reasons.
Although there are some third parties solutions available on the market and although Microsoft will certainly come soon with the Next Generation Video Portal solution for Office 365, let me share with you a high level design of the solution I’ve implemented.
This could give you ideas on how to deal with rich media in general but some aspects of this architecture could be reused for something totally different. The originality of this implementation is that I’m leveraging CORS with Azure and an extension of the out of the box REST endpoints to deal with WAMS/Storage calls.
I’ve seen some other implementations on the web but they either don’t leverage CORS, either requires you to trust an external service where your Azure keys are transmitted which is quite scary IMHO.
So, since what I did isn’t something I’ve seen during my googling, I think it’s worth sharing it. I’m not going to deliver the code since I was paid to write it 🙂 but will more focus on showing the main aspects of the architecture. Before going further, if you’re looking for a pure onilne solution, don’t keep reading since the SharePoint part of my design is only possible with an on-premises setup.
Here is a high level schema of the architecture.
A REST extension is ideal in that scenario since it offers the advantage to be callable from SharePoint Apps in a secure way and requires no specific action. If I had developed a custom web service instead, it would have been more challenging to call it from within an App in a secure way (I’ve blogged many posts about that too).
- 2) The video is uploaded directly via AJAX using a Shared Access Signature (SAS) delivered by WAMS in step 1. That SAS URL targets directly an Azure Storage Account. Unlike WAMS, Storage Accounts can be CORS enabled (google azure storage+cors to see how that works). Using the HTML5 File API, I’m chuncking the videos in smaller pieces and send each piece to the Storage Account right from the client machine. Chuncking offers a better control on error handling since in case a transient error occurs, no need to resend the entire video but just the last chunk. The big advantage of using CORS here is that you’re garanteed to be scalable. You don’t need a proxy that would probably be a bottleneck soon or later when the workload increases, especially when dealing with rich media. Here, you rely directly to the Azure Storage Account capability of dealing with incoming requests. I guess you can sleep like a baby with regards to this
- 3) Once the video has been uploaded, the Video App sends an Encoding Order to WAMS via the REST endpoint _api/wams. This encoding order is added to an Azure Queue. I’m using two Azure Storage Queues. The “ToBeProcessed” queue and the “Processed” queue. As you guessed it, the system adds a new message to the ToBeProcessed Queue whenever a new video gets uploaded. The message contains metadata information about the video & the expected encoding output format. Queues are ideal in this scenario since the kind of messages I’m dealing with are rather small and queues are well designed to handle concurrency when writing/reading into them.
- 4) An on-premises Windows Service checks the ToBeProcessed Storage Queue to trigger and monitor encoding progress in WAMS. When a video encoding job is complete, it adds a new message to the Processed Storage Queue
- 5) A SharePoint Timer Job (minutes) checks the Processed Storage Queue. For each message, it notifies users via Lync (they receive a message in their Lync Client) and via e-mail about video availability with a link to that video. The job creates a new item to the target out of the box Asset Library that contains only a link to the player page.
For the player, I inspired myself from the excelent post from Richard DiZerega on how to implement a Corporate Youtube into SharePoint.
The big difference between his architecture & mine is the usage of CORS. He was using a proxy to upload the videos and again, I’m pretty sure that handling video upload in the worker process is probably not very scalable. I must say that at the time he wrote his blog post, it was not possible to enable CORS at Azure level. Another major difference is that he was using an Auto-Hosted App while I’m using a SharePoint-Hosted App.
So, as a recap, techniques & technologies used in my implementation were:
- REST extension that is easily callable from Apps (and outside Apps too of course) to deal with WAMS (not CORS enabled)
- CORS to upload videos, ensures scalability and eliminate the need of a proxy. HTML5 File API to deal with drag & drop of files, chunks etc..
- Reuse of the OOTB Asset Libraries (inspired from Richard’s post) and a little of jQuery magic to enable Videos preview right from the Asset Library
- WAMS to deal with encoding and also for scalability reasons. You need more encoding power, you can get dedicated Reserved Units, just pay for it 🙂
- Azure Storage Queues to treat everything asynchronously and therefore, being more scalable as well
- Azure Storage to deal with almost limitless storageIn the end, users have the impression that videos are stored in SharePoint and the user experience is not far different from what you have with the usual Asset Library as you can see here:
At last, in order to ensure a good streaming performance, you can work with companies such as Akamai that are specializing in offering CDN services. They will ensure your streaming is optimal and that the path between the client machine and the video is the fastest possible route.