Network Monitoring with VPC Flow Logs

Besides monitoring individual resources, network monitoring is a fundamental task in IT operations. Google VPC gives you flow logs for this that can be enabled on any subnet to log information about transmitted and received network packets and is pure log data for now. Nevertheless, it can be interesting for many use cases to create your own metrics on the basis of these logs (e.g., how often packets were dropped on a network).

VPC Flow Logs are easily enabled by switching to Networking | VPC network | VPC networks in the navigation menu on the left and clicking on the subnet to be configured. When you get there, first switch to Edit mode at the top and then enable the flow logs at the bottom. Once you select On , you can set the aggregation period as well as the sample rate. For most scenarios, the default settings of 5 seconds for aggregation and the sample rate of 50 will be good choices. The latter defines how many values are forwarded to logs. What is also interesting is the estimate below this of how many megabytes per day are aggregated for the subnet. The individual data itself can be viewed in the Logging section.

To select the logs, select Subnetwork in Resource type on the left, choose compute.googleapis.com/vpc_flows as the log name, and then filter by the desired subnet. If you want to evaluate the data with the help of SQL commands, you have to switch logging to legacy mode and then set up a log sink. Next, configure BigQuery – the serverless data warehouse from GCP – as the target. Of course, you can also work with log-based metrics by selecting the Logs-based metrics page in the menu on the left and then setting up a metric for the desired filter.

Conclusions

Google offers a comprehensive set of monitoring tools in its GCP platform that have now reached a high level of maturity. Therefore, efficient monitoring can be set up for the public cloud without the use of third-party tools.