Cloud Storage Pattern

Mitesh N
2 min readDec 11, 2020

Background:

This is very basic architecture solution, one of the simplest and would be useful for those who are just starting with AWS/Cloud solutions. I had helped a customer with web storage issue. They were delivering large files, — video files, high-definition image files, and zip files, from a single web server which was becoming a problem in terms of network load. Ideally, you would distribute the load through multiple web servers to reduce the load on the network– but this involves locating the large files on multiple servers, which is a problem in terms of cost.

Cloud Solution/Pattern:

You can solve the problems of network loads and disk capacities of the web servers by locating the large files on cloud storage such as Amazon S3 and delivering the files directly from there. Objects that are stored in cloud storage can be accessed directly by users if set to being public. This enables delivery from cloud storage, thereby reducing the network load of the web server, and also eliminates the need to copy data between virtual servers in synchronizing the delivery files. Further the cloud storage is pretty cheap in terms of costs.

Architecture / Implementation:

Place the content to be delivered in Amazon Simple Storage Service (S3), and enable user downloading directly from the S3.

  • Create a “bucket” on the S3 storage, and upload the static content — image ,video, compressed files, or any similar that needs to be published.
  • Set this content to be public, enabling user access. When set to be public, a URL will be issued for each individual content object. You now have two options
  1. Provide, a URL to the user, that has been issued, such as http://(bucketname).s3.amazonaws.com/(filename)*, or
  2. Create a link on a webpage.

Advantages

  • The use of S3 eliminates the need to worry about network loads and data capacity.
  • S3 performs backups in at least three different datacenters, and thus has extremely high durability.
  • Because a URL is issued for each content object, the files can be used for a broad range of purposes, such as file sharing, merely through placement on S3.

Caution

Because the content that is delivered requires independent Domain Name System (DNS) naming in S3, the DNS name of the main site cannot be used as-is. For example, if the main site is “www.example.com," then the content on S3 requires a different DNS name, such as “data.example.com,” or the like. Because of this, you may need to change link destinations in the HTML files that have already been constructed.

--

--