Waste management is a major problem for every country. With huge increase in population density large amounts of waste gets produced causing maximal environmental damage. Environmental Protection being high priority makes solid waste management a very important highly expensive and necessary task. Here is an excerpt of a system that we recently worked on that will optimize the cost of operation of any SWM service providers. The project was done for a large SWM provider based out of Mid-West, USA. In this project, each residential dumpster was equipped with an RFID tag which serves as a unique identifier. Each pick-up truck has an RFID receiver and a GPS installed. A heartbeat of truck’s current position along with time stamp is sent to a central server for storage. When the truck collected the waste from a dumpster, its id along with the time and position is also stored. To handle the huge amount of data that is streamed, we used Amazon Web Services
The whole program was deployed on multiple EC2 instances. DynamoDB and elastic cache are used to store the data that would help instances to avoid duplication. In order to fetch the data at high speeds and along with preventing data loss, each instance used SNS to publish data into multiple queues which is then pulled by another set of multiple EC2 instances for processing and cleaning. The processed data is added to S3 buckets as a CSV file. Amazon data pipeline is configured to move CSV data from S3 bucket to Redshift. Once the data is transferred to the data warehouse it was visualised and analytics was generated using Tableau.
Amazon EC2– Amazon Elastic Compute Cloud is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.
Amazon SNS – Amazon Simple Notification Service is a fast, flexible, fully managed push messaging service.
Amazon SQS – Amazon Simple Queue Service is a fast, reliable, scalable, fully managed message queuing service. SQS makes it simple and cost-effective to decouple the components of a cloud application. It can be used to transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available.
Amazon Redshift – Amazon Redshift is a hosted data warehouse part of the larger cloud computing platform Amazon Web Services. It is a fast, fully managed, petabyte-scale data warehouse service that makes it simple and cost-effective to efficiently analyze all your data using any existing business intelligence tools.