Imagine the situation: you browse your Amazon S3 bucket and suddenly discover that some files are missing. Since Amazon S3 offers high durability, it leaves almost no chance for these files to disappear due to a system failure or disaster. Apparently, they were deleted by a user. How to find out who did that?
Amazon CloudFront has been offering access logs for HTTP distributions for quite some time. Now this feature is extended to work with streaming distributions as well. Amazon team has designed CloudFront access logs for streaming to work in a very similar fashion to the existing access logs for HTTP distributions. Continue reading
In our previous post, we talked about how to enable Amazon S3 server log. In this post, we will show you some of the cool CloudBerry Explorer features that allow you to view Amazon S3 server log in a human-readable form.
How to generate a report
To generate a report simply right-click the bucket and choose View Server Access Log from the context menu. This option will be available if you have enabled the log for the bucket previously. You can click to the Logging Setting to enable the server log.
What reports are available
Reports are available in a tabular form and a chart form. Reports are not available instantly. When log files are loaded and a report is generated you will experience a delay that depends on the volume of your log files and see a progress.
There are two charts available:
1.the total number of requests per day
2.the total traffic per day (shown on the screen)
CloudBerry Explorer provides you only with basic means with several built-in reports for analyzing your S3 server log. If you want to have more advanced and more flexible log analysis I suggest that you look for other tools such as SiSense Prism Viewer with Amazon S3 Dashboard. Check out our full review of this product here.
You can also check out an online service called S3Stat that generate reports from your log files using Webalizer and delivers them to your S3 bucket. These guys have also recently added CloudFront log support. Check out their latest blog post on the subject Code on the Road: Cloudfront Analytics from S3stat
CloudBerry lab is trying to keep up with the fascinating pace of the innovations that are rapidly introduced by Amazon Web Services team. We are excited about the introduction of CloudFront Access log and glad to announce that we support it in CloudBerry Explorer
What is CloudFront Access logging?
CloudFront access log helps you to understand what kind of request served by your CloudFront distributions over time. Access logs are activity records that show you details about every request delivered through Amazon CloudFront. It is similar to the log file that popular web servers such as Apache or MS IIS generate. With the log file, you can get a feel of the nature of a request for your content.
How to turn on Amazon CloudFront log
Just select the bucket configured for CloudFront and click the logging on the toolbar. Check Use CloudFront Access Logs option; specify the prefix and the bucket where you want the log files to be written.
Note: you can specify somebody a bucket from another Amazon S3 account; just make sure you have proper permissions set. Just make sure that the owner of the CloudFront distribution has a write access to the bucket.
How to view CloudFront logs
Once you turn on your logs they will start coming up in the bucket you configured it for. Logs will typically be written every hour. Files are coming in and they are compressed using GZip to save some space. CloudFront Access Logs will use the W3C extended format (http://www.w3.org/TR/WD-logfile.html). An example of the content of your bucket where the log files are located is shown below: CloudBerry Explorer allows you to view files in the raw format.
Or you can switch to the log viewer format that will help you to view your log files in the grid, sort, and filter. It
You can also view a simple chart that shows you the number of requests/ number of bytes per day.
Another useful chart shows the breakdown of the activity by CloudFront edge locations.
Is there a more advanced way to analyze CloudFront logs
Yes! There are multiple tools out there that will help you to slice and dice CloudFront logs. Good Data provides Amazon CloudFront Analytics online. If you incline to a desktop based product I would suggest SiSense Prism Viewer with Amazon S3 dashboard.
With the addition of CloudFront access log support in CloudBerry lab demonstrates ongoing commitment to make the most comprehensive solution for Amazon S3 and CloudFront services. CloudBerry Explorer is becoming a compelling tool that helps users to accomplish their tasks in the most efficient way.
If you are wondering what kind of request is coming to your bucket and what actually you pay money for you may consider turning on Amazon S3 logging to record the requests coming to your bucket. This is a simple task that CloudBerry Explorer freeware will make it even easier.
A bucket to keep log files
Let’s create a bucket called cloudberry.log to keep your log file. You don’t necessarily have to create a new bucket for your log files, but I think it is a good idea to have a separate bucket for the purpose to make things better organized.
Let’s imagine we want to configure a logging for a bucket called images.cloudberrylab.com.So, all you have to do is to select the bucket and to click Logging button on the toolbar.
Now check “Use logging” checkbox and choose the bucket where you want the log files to be written in the dropdown list. In our example it is cloudberry.log.
Click ok and you are done. According to Amazon documentation, log file doesn’t appear immediately. It may take up to 24 hours for the log file to be created.
Now you are asking yourself: what can I do with the log files, how I can view them? What format are they stored in? Read our second blog post in this series on analyzing the log files.
A log file can be written to the bucket in the same geographical location. In other words, if you are logging the access to the bucket in the US you will have to assign another bucket in the US to keep that log.
Also, notice that log file will add up to your monthly bill (although insignificantly) as you will need some additional S3 storage for your log files. The number of log files will grow over time so don’t forget to clean them up regularly.
S3 Log accuracy
One thing worth mentioning that even though Amazon will make the best effort to make the log as accurate as possible it doesn't guarantee its complete accuracy. In a rare situation, certain requests will not make it to the log. But as Amazon explains S3 log is designed to give a general idea of how your bucket is being used.