Automated Log Data Processing and Integration System

Objective:

To automate the processing of generating daily traffic logs for the server.

Responsibilities:

Developed a Shell script to parse Apache logs into CSV files and a Node.js script to convert these files into JSON and send them to a Google App Script WebApp. For China servers, an AWS API Gateway and Lambda function was implemented to bypass the China firewall. Integrated the workflow with Google Sheets using Google App Script to dynamically store and organize data for reporting.

Technologies:

Shell Scripting, Node.js, AWS Lambda, AWS API Gateway, Google App Script, Google Sheets.