Result

Result Translation
- GCP aichip 비용 보고서 (2024-11-24) == GCP project name aichip’s costs daily Reports (2024-11-24)
- 이번달 누적 금액 == This month total costs
- 전일 == Yesterday
- SKU 전일 사용 금액 (1000원 이상만 표시) ( Costs of each SKU , which is Over 1$)

Required IAM

How to do?

1. Set Billing Export at Billing Section

2. Set BigQuery dataset

💡 It needs a day data exported and saved into the BigQuery

3. Use Slack Webhook

Copy your slack webhook url

4. Make Serviceaccount and download into json to use python code with slack webhook

Sample Codes

```json
{
  "type": "service_account",
  "project_id": "[your project id],

setup slack webhook url

SERVICE_ACCOUNT_PATH = 'project id.json'
SLACK_WEBHOOK_URL = "https://hooks.slack.com/services/...[YOUR SLACK URL]"

BigQuery serviceaccount setups and SQL


def get_gcp_costs_by_sku():
    credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_PATH)
    client = bigquery.Client(credentials=credentials, project=PROJECT_ID)
		...

    sku_query = f"""
        SELECT
            sku.description AS sku_description,
            SUM(cost) AS total_cost,
            SUM(usage.amount) AS total_usage,
            ANY_VALUE(usage.unit) AS usage_unit
        FROM
            `{PROJECT_ID}.[YOUR DATASET NAME]`
        WHERE
            usage_start_time >= '{start_of_month}'
            AND usage_start_time < '{today}'
        GROUP BY
            sku_description
        HAVING
            total_cost > 100
        ORDER BY
            total_cost DESC
    """

    sku_result = client.query(sku_query).result().to_dataframe()



### CronJob with k8s

- containerize simply

```json
FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "billing.py"]

apiVersion: batch/v1
kind: CronJob
metadata:
  name: mobidoc-billing-report
  namespace: default 
spec:
  **timeZone: 'Asia/Seoul' # only work in k8s 1.27+ 
  schedule: "0 9 * * *"**
  jobTemplate:
    spec:
      template:
        metadata:
          annotations:
            sidecar.istio.io/inject: "false" #istio 깔려있다면 필요에 따라 설정
        spec:
          containers:
          - name: mobidoc-billing
            image: registry.gocap.kr/etc/mobidoc-billing:0.0.2
            imagePullPolicy: Always
            env:
              - name: TZ
                value: "Asia/Seoul" 
          restartPolicy: OnFailure

Key Learnings and Insights

Through this project, I explored various cloud cost management and automation strategies by integrating GCP Billing, BigQuery, Kubernetes CronJobs, and Slack Webhooks.

Automated Cloud Cost Reporting
- Designed a system to extract, store, and analyze billing data using BigQuery, allowing real-time cost monitoring without relying on the GCP console.
- Gained hands-on experience with SQL-based cost aggregation and filtering strategies to extract meaningful insights.
Kubernetes CronJobs for Scheduled Automation
- Implemented Kubernetes CronJobs to automate daily billing reports, reducing manual monitoring efforts.
- Explored timezone configurations and Istio sidecar management in K8s batch processing environments.
Slack Webhook Integration for Real-Time Notifications
- Developed a Slack bot to automatically send cost reports, improving operational visibility.
- Learned best practices for integrating cloud monitoring with collaboration tools, enabling faster response to cost anomalies.

This project strengthened my expertise in cloud billing analysis, Kubernetes batch processing, and automated operational workflows, providing valuable insights into optimizing cloud cost management and DevOps automation.

Loading Project...