Azure Storage offers many useful features and services that make integration into existing systems easier. New solutions can take advantage of these features on an architectural level. Knowing which common patterns are natively supported by Azure Storage can radically accelerate development. Here is an overview of them.
Blob Storage / Blob Service / Containers
Shared Access Tokens
A container can delegate certain permissions via a SAS (shared access signature) token to JavaScript code (for example). Every token is signed by a private key. There are three methods of generating the token, each of which provides permissions in a different way:
Role-based permissions of a user authenticated by Azure Active Directory.
Individual permissions delegated by users of Azure Active Directory.
Individual permissions based on one of two keys of the storage account, which can be manually regenerated.
These tokens can be easily generated dynamically by code, so it makes sense to limit their duration to a specific time. When the token somehow leaks, the attacker has a limited time to perform a malicious action.
Immutable Storage
Many papers must be archived due to legal reasons. Immutable Storage is a digital equivalent of a document archive. The container can be locked with an Immutable Blob Storage policy. Locked blobs cannot be deleted, modified, or moved. There are two kinds of locks – Time-based retention and Legal hold. Time-based retention holds the lock for a specified period. Legal hold is an assigned tag that locks the container or blob.
Metadata
A blob can hold additional key-value pairs of data. A typical example is the Content-Type, which is served as an HTTP header. However, even custom metadata is contained in HTTP headers with an X prefix. It is extremely helpful in plenty of scenarios because JavaScript has the capability to read this metadata.
Index Tags
Blob index tags provide a built-in capability to list blobs by custom attributes. A blob’s tag can be set during or after upload. Each blob can have up to 10 index tags. Additional pricing is based on the monthly average number of index tags in the storage account.
Hierarchical Namespace
Every (general purpose v2) storage can be upgraded to Data Lake Gen2 storage. This migration allows us to take advantage of the hierarchical namespace. More specifically, it brings us:
- Efficient querying of subfolders.
- Much faster renaming or moving of blobs.
- Atomic operations with the ABFS driver over the DFS endpoint.
- Granular POSIX-compliant security.
Soft Delete
Containers or blobs don’t always have to be permanently deleted. It is possible to set a period that delays actual deletion. During this time, those items are hidden and can be restored. When this period ends, permanent deletion occurs automatically.
Access Tier
To achieve cost savings, data can be distributed among different storage accounts with a specific access tier that best fits the data’s nature. Azure currently offers four kinds of tiers:
- Hot tier – highest storage costs, lowest access cost.
- Cool tier – about 30% less expensive than the hot tier but write or read operations are more expensive. If you delete a blob before it becomes 30 days old, you must pay the early deletion fee.
- Cold tier – about 80% less expensive than the hot tier but write operations are twice as expensive as the cool tier. The early deletion fee will apply within 90 days.
- Archive tier – about 95% less expensive than the hot tier but the delay between data request and delivery is in hours. The early deletion fee is applied if you delete a blob before it becomes 180 days old. Retrieving an archived blob can take up to 15 hours. If you pay a little extra for the priority operation, it can take less than 1 hour.
Blobfuse
Blobfuse allows access to block blob data in your storage account through the Linux file system. It is a virtual file system driver for Ubuntu, Debian, SUSE, CentOS, Oracle Linux, and RHEL distributions.
Inventory
Inventory reports are a tool to get an overview of all your data within a storage account. Reports are created periodically – daily or weekly. They are in CSV format and are automatically stored in a specific container.
Snapshots
A snapshot is a read-only copy of a blob taken at a certain point in time. Snapshots, unlike versions, are created manually. Snapshots of blobs in the Archive tier are not supported.
Versions
Azure Storage can automatically save a previous version every time a blob is modified (or deleted). Previous versions can be listed via SDK (or Azure Portal). Older versions can be stored in a different access tier than the current (propagated) version.
Task: Delete Old Blobs
This feature, currently in preview, can simplify many cloud solutions and save many lines of code. It deletes all blobs in a specific container older than a given period.
Lifecycle Management
Blobs that haven’t been modified for a specific period can be automatically deleted or moved to cool storage or archive storage. The rule applies to the whole storage or to a specific subset (excluding append blobs) based on the blob’s name or metadata.
Table Storage / Table Service
Cosmos DB Table API
Azure Cosmos DB is accessible in the same way as traditional Table Storage (with the newer Azure Table SDK). An entity in Azure Storage can be up to 1 MB in size. An entity in Azure Cosmos DB can be up to 2 MB in size.
Queue Storage
Infinite TTL Interval
The maximum message lifetime was always 7 days. It is now possible to opt-in for an immortal message that never expires.
File Shares / File Service / Azure Files
Large File Shares
The maximum file size is 5 TB. If you activate the large file shares option, this limit grows to 100 TB. However, this action is irreversible. A minor side effect is that this storage cannot be geo-redundant, so it is limited to a single region.
Soft Delete
If a mapped network drive is connected via the SMB protocol, deleted files can be restored via the Azure Portal (or a PowerShell script). The maximum retention period is 365 days. Soft delete for NFS or SFTP is supported by Azure Data Lake Storage.
Premium Performance
Basic storage accounts are physically on HDDs. Premium accounts are located on SSDs, which provide much higher IOPS and much lower latency. A premium storage account can host a premium tier file share. It has cheaper transaction costs compared to the standard tier. The IOPS and throughput are based on the provisioned size. On the other hand, a premium file share does not support any form of geo-redundancy.
Storage Access Policies
If you authorize access to Storage, Table, or Queue via Azure Active Directory, you can assign certain roles to security principals (users, groups, or application services). A role permits or denies specific actions:
- Storage – read, add, create, write, delete, list.
- Table – read, add, update, delete.
- Queue – read, add, update, process.