Exploring XML In Snowflake
In this article we will see how to load and use XML files in snowflake. Code used in this article can be found here. …
In this article we will see how to install AWS SAM in a docker container and to create an AWS Lambda using AWS SAM.
Download and install docker for your platform. Click here for instructions
Create an AWS account. Click here for instructions
Create an IAM user in AWS with required access. See here for more details about the required permissions.
Create an access key for this IAM user. The access key will be used by SAM to deploy the resources.
Open a new terminal and cd into aws-tools
directory. This directory contains Dockerfile and docker-compose.yml to bring up a docker container with AWS SAM.
Create a copy of .env.template
as .env
Update the .env
with the access key from previous step
Start aws-tools container by running
docker-compose up -d
Validate the container by running
docker ps
SSH into the container by running
docker exec -it aws-tools /bin/bash
Validate AWS CLI by running
aws --version
Validate AWS SAM by running
sam --version
Validate your AWS profile by running
aws configure list
You can also override the profile OR create a new profile at any point by running aws configure
OR aws configure --profile <profile-name>
cd into your preferred directory
cd /C/
Create SAM project by running and following the guided instructions
sam init
This can be also done in non interactive mode by invoking the command
sam init --name sam-lambda-api --runtime python3.9 --dependency-manager pip --app-template hello-world --no-interactive
cd into the SAM project directory and validate SAM project by running
sam validate
Build SAM project by running
sam build
Invoke SAM project locally by running
sam local invoke --container-host host.docker.internal
Start Lambda locally by running
sam local start-lambda --container-host host.docker.internal --host 0.0.0.0
Invoke Lambda by running below command from a new session
aws lambda invoke --function-name "HelloWorldFunction" --endpoint-url "http://localhost:3001" --no-verify-ssl lamda_output.txt && cat lamda_output.txt
Start API locally by running
sam local start-api --container-host host.docker.internal --host 0.0.0.0
Validate local API by invoking endpoint from a browser OR postman
http://localhost:3000/hello
Deploy SAM project by running below command
sam deploy --guided
OR
sam deploy --stack-name sam-lambda-api --s3-bucket <bucket-name>
Validate the SAM deployment status from the CloudFormation console.
Navigate to the output tab of CloudFormation template to collect the API endpoint.
Validate API by invoking API endpoint from a browser OR postman
Validate SAM/Lambda logs by running below command
sam logs -n HelloWorldFunction --stack-name sam-lambda-api --tail
You can find more information and examples about filtering Lambda function logs in the SAM CLI Documentation.
SAM/Lambda logs can be also validated from CloudWatch Log groups for the Lambda created by the SAM project
Delete the SAM application by running
aws cloudformation delete-stack --stack-name sam-lambda-api
Delete the docker container from a new terminal, cd into aws-tools
directory and run the below command to delete the docker containers and related volumes
docker-compose down --volume --remove-orphans
In this article we will see how to load and use XML files in snowflake. Code used in this article can be found here. …
In this article we will see how to stream data from kafka to snowflake using S3 and Snowpipe. Code used in this article …