Tag Archives: Log

How to Improve Security of S3 Buckets Using S3 Access Logs

Amazon S3 bucket logging: enable, read and access S3 logsImagine the situation: you browse your Amazon S3 bucket and suddenly discover that some files are missing. Since Amazon S3 offers high durability, it leaves almost no chance for these files to disappear due to a system failure or disaster. Apparently, they were deleted by a user. How to find out who did that?
Continue reading

CloudBerry Explorer: Azure Storage Analytics Log Viewer

Just about a week ago we introduced CloudBerry Explorer for Azure Blob Storage the support for Azure Storage Analytics. In this follow up release we are introducing a Log Viewer. This is a simple tool that helps you look over the log file in a human readable format. Continue reading

CloudBerry Explorer: Windows Azure Storage Analytics Support

It has been awhile since we introduced CloudBerry Explorer client for Microsoft Azure Blob storage service. We designed it as an Azure storage browser that helps managing your data and now we are proud to announce a release which introduces a new Azure Storage Analytics tool.

Continue reading

How to Configure Amazon CloudFront Streaming Log with CloudBerry Explorer

Amazon CloudFront has been offering access logs for HTTP distributions for quite some time. Now this feature is extended to work with streaming distributions as well. Amazon team has designed CloudFront access logs for streaming to work in a very similar fashion to the existing access logs for HTTP distributions. Continue reading

How to check Amazon S3 server log using CloudBerry S3 Explorer

In our previous post we talked about how to enable Amazon S3 server log. In this post we will show you some of the cool CloudBerry Explorer features that allow you to view Amazon S3 server log in a human readable form.

How to generate a report

To generate a report simply right click the bucket and choose View Server Access Log from the context menu.This option will be available if you have enabled the log for the bucket previously. You can click to the Logging Setting to enable the server log.

What reports are available

Reports are available in a tabular form and a chart form. Reports are not available instantly. When log files are loaded and a report is generated you will experience a delay that depends on the volume of your log files and see a progress.

There are two charts available:

1.the total number of requests per day

2.the total traffic per day (shown on the screen)

Conclusion

CloudBerry Explorer provides you only with basic means with several built-in reports for analyzing your S3 server log. If you want to have more advanced and more flexible log analysis I suggest that you look for other tools such as SiSense Prism Viewer with Amazon S3 Dashboard . Check out our full review of this product here.

You can also check out an online service called S3Stat that generate reports from your log files using Webalizer and delivers them to your S3 bucket. These guys have also recently added CloudFront log support. Check out their latest blog post on the subject Code on the Road: Cloudfront Analytics from S3stat

Note: this post applies to CloudBerry Explorer 1.5.1 and later.
CloudBerry S3 Explorer is a Windows product that helps managing Amazon S3 storage and CloudFront.
Related products:
Like our products? Please help us spread the word about them. Learn here how to do it.

How to Conduct Amazon CloudFront Logs With CloudBerry Explorer

CloudBerry lab is trying to keep up with the fascinating pace of the innovations that are rapidly introduced by Amazon Web Services team. We are excited about the introduction of CloudFront Access log and glad to announce that we support it in CloudBerry Explorer

What is CloudFront Access logging?

CloudFront access log helps you to understand what kind of request served by your CloudFront distributions over time. Access logs are activity records that show you details about every request delivered through Amazon CloudFront. It is similar to the log file that popular web servers such as Apache or MS IIS generate. With the log file you can get feel of the nature of request for your content.

How to turn on Amazon CloudFront log

Just select the bucket configured for CloudFront and click the logging on the toolbar. Check Use CloudFront Access Logs option; specify the prefix and the bucket where you want the log files to be written.

Note: you can specify somebody a bucket from another Amazon S3 account; just make sure you have proper permissions set. Just make sure that the owner of the CloudFront distribution has a write access to the bucket.

How to view CloudFront logs

Once you turn on your logs they will start coming up in the bucket you configured it for. Logs will typically be written every hour.Files are coming in and they are compressed using GZip to save some space. CloudFront Access Logs will use the W3C extended format (http://www.w3.org/TR/WD-logfile.html). An example of the content of your bucket where the log files are located is shown below: CloudBerry Explorer allows you to view files in the raw format.

Or you can switch to the log viewer format that will help you to view your log files in the grid, sort and filter. It

You can also view a simple chart that shows you the number of requests/ number of bytes per day.

Another useful chart shows the breakdown of the activity by CloudFront edge locations.

Is there more advanced way to analyze CloudFront logs

Yes! There are multiple tools out there that will help you to slice and dice CloudFront logs. Good Data provides Amazon CloudFront Analytics online. If you incline to desktop based product I would suggest SiSense Prism Viewer with Amazon S3 dashboard.

Conclusion

With the addition of CloudFront access log support in CloudBerry lab demonstrates ongoing commitment to make the most comprehensive solution for Amazon S3 and CloudFront services. CloudBerry Explorer is becoming a compelling tool that helps users to accomplish their tasks in the most efficient way.

CloudBerry S3 Explorer is a Windows product that helps managing Amazon S3 storage and CloudFront.
Related products:
Like our products? Please help us spread the word about them. Learn here how to do it.

How to Switch on Amazon S3 Bucket Logging With CloudBerry Explorer (Part 1)

If you are wondering what kind of request is coming to your bucket and what actually you pay money for you may consider turning on Amazon S3 logging to record the requests coming to your bucket. This is a simple task that CloudBerry Explorer freewarewill make it even easier.

A bucket to keep log files

Let’s create a bucket called cloudberry.log to keep your log file. You don’t necessarily have to create a new bucket for your log files, but I think it is a good idea to have a separate bucket for the purpose to make things better organized.

Configure logging

Let’s imagine we want to configure a logging for a bucket called images.cloudberrylab.com.So, all you have to do is to select the bucket and to click Logging button on the toolbar.

Now check “Use logging” checkbox and choose the bucket where you want the log files to be written in the dropdown list. In our example it is cloudberry.log.

Click ok and you are done. According to Amazon documentation log file don’t appear immediately. It may take up to 24 hours for the log file to be created.

What next?

Now you are asking yourself: what can I do with the log files, how I can view them? What format they are stored in? Read our second blog post in this series on analyzing the log files.

Final notes

Log file can be written to the bucket in the same geographical location. In other words if you are logging the access to bucket in the US you will have to assign another bucket in the US to keep that log.

Also notice that log file will add up to your monthly bill (although insignificantly) as you will need some additional S3 storage for your log files. The number of log files will grow over time so don’t forget to clean them up regularly.

S3 Log accuracy

One thing worth mentioning that even though Amazon will make the best effort to make the log as accurate as possible it doesn't guarantee its complete accuracy. In rare situation certain requests will not make it to the log. But as Amazon explains S3 log is designed to give a general idea how your bucket is being used.

Related products:
Like our products? Please help us spread the word about them. Learn here how to do it.