CloudCurls

Introduction When working with large datasets, execution time and efficiency comes into play. Traditional methods of extracting data from the relational databases into Python often involve loading everything into memory, which can be painful and very slow. That’s where connectorX and DuckDB come in handy. Together, they make data extraction and analytics in python very  fast and memory-efficient . What is ConnectorX? ConnectorX is an open-source library built to load data from databases directly into pandas, Polars, or NumPy efficiently. Instead of fetching row by row via  psycopg2  or  sqlalchemy ConnectorX p arallely fetch chunks of data and stream them directly into Python. Supports many databases: MySQL, SQLite, PostgreSQL, SQL Server, BigQuery, Snowflake, and many more. What is DuckDB? DuckDB is an in-process SQL OLAP database. Can query CSV, Parquet, JSON, Arrow datasets, and even pandas/Polars DataFrames. Works directly inside Python and R. Data pro...

Introduction In modern cloud-native applications, managing sensitive information like API keys, database credentials, any third party service credentials and other secrets securely is a top priority. Hardcoding secrets into application code, environmental variables in lambda functions or configuration files can lead to serious security vulnerabilities and operational risks and this is where AWS Secrets Manager comes in—a fully managed service that enables you to store, retrieve, and rotate secrets securely. When combined with AWS Lambda , Secrets Manager allows you to build powerful serverless applications that access secrets dynamically during the runtime, without ever exposing them in your codebase. In this blog, we'll explore how to integrate AWS Secrets Manager with Lambda functions, ensuring your application remains secure, scalable, and maintainable. Whether you're accessing a database, calling a third-party service, or simply avoiding secret sprawl, this guide will wal...

Introduction When working with AWS Lambda functions in Python, especially in database-heavy applications, you often run into deployment package size limits or performance issues due to repeated bundling of common libraries like psycopg2 for PostgreSQL, python-oracledb for Oracle or  mysql-connector-python for MySQL. These database drivers are essential, yet bulky—leading to bloated deployment packages, slower cold starts, and painful debugging across environments. To address this, Lambda Layers offer a powerful solution. Layers allow you to package shared dependencies—such as database drivers—separately and reuse them across multiple functions, simplifying deployment and improving scalability. In this blog, we’ll walk through creating a scalable and reusable Lambda Layer for PostgreSQL or MySQL drivers using Python. You’ll learn not only how to build and deploy these layers, but also best practices to make your architecture more maintainable and efficient in the long run. Whet...

Introduction Securing APIs is essential for protecting sensitive data and ensuring authorized access to your applications. AWS provides a seamless way to achieve this through the integration of Amazon Cognito and API Gateway. Amazon Cognito simplifies user authentication and identity management, while API Gateway serves as a scalable entry point for your backend services. This blog will guide you through securely connecting Amazon Cognito with API Gateway. You'll learn how to configure user pools, set up authorization mechanisms, and test the setup to ensure only verified users can access your APIs. By following this guide, you'll gain valuable insights into building secure and scalable applications using AWS services. Creation of Cognito User Pool Open the AWS Management Console and search for Amazon Cognito. Navigate to User Pools and select Create User Pool. Begin by defining your application settings. For this tutorial, we'll choose the Traditional Web Application type....

Introduction to AWS Key Management Service (KMS) In today’s digital landscape, safeguarding sensitive data is critical. As organizations increasingly migrate from legacy systems to the cloud, robust encryption and key management solutions become essential to secure data at rest as well as in transit. AWS Key Management Service (KMS) is a fully managed service that helps you create, control, and manage encryption keys securely across AWS services and your applications.  Whether you’re protecting customer information, securing application secrets, or encrypting sensitive documents, AWS KMS provides a scalable and cost-effective solution that integrates seamlessly with other AWS services. In this blog, we will explore how AWS KMS works, its core features, and how it can enhance your security posture in the cloud. Additionally, we will walk through practical use cases and best practices for implementing KMS in your AWS environment. Creating Customer-Managed Keys There are essentially t...

Setting Up Amazon SES destinations Amazon Simple Email Service (SES) allows users to send, receive, and monitor emails at scale. One powerful feature is its ability to send event data (e.g., bounces, complaints, and delivery notifications) to various destinations. In this tutorial, we'll guide you through setting up SES destinations with the following services: Amazon Notification Service SNS Kinesis Data Firehose Amazon Pinpoint SNS as Destination Create an SNS topic Log in to the AWS Console and search for Simple Notification Service (SNS). Select it from the results. Navigate to the Topics section and click Create Topic. Choose Standard as the topic type, optionally add tags, and then click Create Topic to complete the setup. Type of SNS topic Configure the destination in SES Navigate to Simple Email Service (SES) and choose the configuration set where you want to add a destination. Click Add Destination, select the desired event types, and proceed to the next step. Under Destin...

  If you're seeking a reliable and cost-effective solution for email communication then AWS Simple Email Service (SES) is an excellent choice. This cloud-based service enables you to efficiently send and receive emails with ease. Prerequisites This tutorial offers a hands-on demonstration. Ensure you have an active AWS account to follow along AWS Login . 1.  What is AWS SES AWS Simple Email Service is a cloud-based service provided by AWS that enables the business and developers to send, receive, and manage email campaigns securely and cost-efficiently. Use Cases: Transactional email notifications for applications. Marketing Campaigns Newsletters System alerts Automated Reports 2.  Why Use AWS SES? Reliable and scalable infrastructure Cost-effective compared to third-party email services Integration with other AWS services like Lambda, CloudWatch, SNS, EventBridge and Pinpoint. Built-in tools to track email events (sent, deliveries, open, click, bounces, complaints) Secur...