What is Amazon Redshift?

What is Amazon Redshift?

Amazon Redshift is a fully-managed petabyte-scale cloud based data warehouse product designed for large scale data set storage and analysis. It is also used to perform large scale database migrations.

Redshift’s column-oriented database is designed to connect to SQL-based clients and business intelligence tools, making data available to users in real time. Based on PostgreSQL 8, Redshift delivers fast performance and efficient querying that help teams make sound business analyses and decisions.

What is a Redshift Cluster?

Each Amazon Redshift data warehouse contains a collection of computing resources (nodes) organized in a cluster. Each Redshift cluster runs its own Redshift engine and contains at least one database.

Is Amazon Redshift a Relational Database?

Redshift is Amazon’s analytics database, and is designed to crunch large amounts of data as a data warehouse. Those interested in Redshift should know that it consists of clusters of databases with dense storage nodes, and allows you to even run traditional relational databases in the cloud.

Is Redshift fully managed?

Redshift is a fully managed cloud data warehouse. It has the capacity to scale to petabytes, but lets you start with just a few gigabytes of data. Leveraging Redshift, you can use your data to acquire new business insights.

Amazon Redshift vs Traditional Data Warehouses

Amazon Redshift is a direct alternative to on-premise traditional database warehouses. Let’s look at how Redshift stacks up to traditional warehousing in the following areas:

• Performance
• Cost
• Scalability
• Security

AWS Redshift Performance

Amazon Redshift is most known for its speed. Redshift delivers the fast query speeds on large data sets, dealing with data sizes up to a petabyte and more. The speed by which Redshift processes data up to these sizes is just simply impossible to attain in traditional data warehousing, making it the top choice for applications that run massive amounts of queries on-demand.

The ability to deliver this level of performance comes with the use of two architectural elements: columnar data storage and massively parallel processing design (MPP). We will delve deeper into these two later.

Redshift Cost

Amazon Redshift is markedly faster than traditional warehousing–but when it comes to choosing tech solutions, organizations are arguably most concerned about cost.

As a cloud-based solution, Amazon Redshift is able to provide high-level performance affordably. IT executives know that traditional warehousing is extremely costly from the beginning, with the initial outlay for hardware possibly costing up to the multi-millions. On the other hand, there are no substantial upfront costs to getting setup and starting with Redshift. Being a fully-managed solution, Redshift has no recurrent hardware and maintenance costs. Database admins can set up data warehouses that can handle massive amounts of data without having to go through the lengthy process of procurement and strategic buy-in from leadership that multi-million-dollar on-premise hardware requires.

Redshift Scalability

Traditional on-premise data warehousing poses quite the challenge in case your data needs increase or decrease.

For traditional warehousing, when organizations’ data needs change, they are forced to have to make another round of costly investments for new hardware purchase and implementation.

Redshift allows for more flexibility and elastic scale. As your requirements change, Redshift can scale up or down instantly to match your capacity and performance needs with a few clicks in the management console.

Cost-wise, on-demand pricing ensures you only pay for what you use. Not being tied down to expensive hardware and lengthy maintenance contracts mean organizations have the liberty to change their minds without having to eat up sunk costs. From a single 160GB DC1.Large node all the way up to multiple 16TB DS2.8XLarge nodes for a petabyte or more of data, you have access to processing power on-demand.

Security in Redshift

Although Amazon Redshift is demonstrably better than traditional warehousing in the abovementioned regards, security remains to be the tipping point for many enterprises–but it’s not because of known security vulnerabilities. The reality is that some still feel concerned about not having their data physically present.

That said, security is a topmost concern for Amazon, knowing this is a salient point in the decision making for warehousing solutions.

Redshift Security Best Practices

Amazon follows the shared responsibility model of security where Amazon is responsible for the security of the cloud, and the organization is responsible for security in the cloud.

• Security of the cloud: AWS protects infrastructure where AWS services run in the cloud. They are responsible for making sure that features and services that can be used securely are available to users. AWS also ensures that security levels are regularly tested and verified as part of AWS compliance.
• Security in the cloud: The security responsibility of organizations using Redshift is determined by the AWS service they use. Organizations are also responsible for other factors like data sensitivity, an org’s own internal requirements, and compliance with laws and regulations.

That said, Amazon Redshift has most security features of the larger Amazon Web Services platform. Credentials and access are granted and managed on the AWS-level through Identity and Access Management (IAM) accounts. Cluster security groups are created and associated with data clusters for inbound access. For orgs that use a private cloud, access through a Virtual Private Cloud (VPC) environment is available as well. Data encryption is also enabled upon cluster creation and cannot be switched from encrypted to unencrypted directly.

For data in transit, Redshift uses SSL encryption to communicate with S3 or Amazon DynamoDB for COPY, UNLOAD, backup, and restore operations.

Amazon Redshift Performance

As mentioned above, Amazon Redshift is able to deliver performance with best-in-class speed due to the use of two main architectural elements: Massively Parallel Processing (MPP) design and columnar data storage. Let’s look at each one and see how they enable fast processing in Redshift.

Redshift’s Massive Parallel Processing (MPP) Explained

Redshift’s Massively Parallel Processing (MPP) design automatically distributes workload evenly across multiple nodes in each cluster, enabling speedy processing of even the most complex queries operating on massive amounts of data. Multiple nodes share the processing of all SQL operations in parallel, leading up to final result aggregation. Users can optimize the distribution of data by locating the data where it needs to be before the query is executed. This is done by choosing the appropriate distribution style, minimizing the impact of the redistribution step.

Redshift Columnar Data Storage Explained

By using columnar storage for database tables, Amazon Redshift reduces the disk I/O requirements, contributing to the optimization of analytic query performance. When database table information is stored in a columnar fashion, the number of disk I/O requests and the amount of data needed to be loaded from disk are reduced. When less data is loaded into memory, Redshift can perform more in-memory processing for executed queries. The amount of time needed to perform a query is reduced using this method compared to when data is stored by row.

Authorizing access to the Redshift cluster

After following the steps, the Redshift cluster is now launched. To connect to the cluster, you need to configure a security group to authorize access. If the cluster is launched in the EC2-VPC platform, follow these instructions from AWS.

Connecting to the cluster and running queries

Now that you have launched a cluster, you may connect to it and start running queries. Running queries can be done in two ways:

1. 1. Connect to your cluster from the AWS Management Console using the AWS Query Editor.
2. 2. Connect to your cluster through a SQL client tool like SQL Workbench/J.

At this point, you can now use your Redshift cluster. You can create tables in the database, upload data to the tables, and try running queries. These activities can be done through the AWS Query Editor or through a SQL client tool of your choice.

How to Monitor Amazon Redshift

Now you know how Amazon Redshift works and why it’s fast and efficient. Still, the best way to know for sure is to see its performance for yourself by monitoring performance. In the next blog posts in this series, we will take a deep dive into how to analyze Redshift queries and how to monitor Amazon Redshift performance with Sumo Logic. Stay tuned.