Category: Azure

  • Handle argumented Docker contrainer registry paths

    In our a recent setup I’ve created, we are using Azure Container Registry to hold a base image, which we use when building our main project.

    I use a docker-compose.yml and Dockerfile in our setup and github actions for deployment.

    The base image is deployed to two container registries, because of access rights on the different environments.

    Basically we have a develop and a production environment.

    The develop environment also has a staging setup.

    When using our base image, we need to have this in both the develop and production container registries. This means, that our Dockerfile needs to have two different urls to the base image, since they cannot access the other environments container registry.

    Usually the develop environment has the following setup:

    # Base path for image
    FROM uniquedevrepo.azurecr.io/projectbase:php7.4
    
    # other docker stuff here...
    
    # expose the webserver
    EXPOSE 8080 443

    And the production environment had the following setup:

    # Base path for image
    FROM uniqueprodrepo.azurecr.io/projectbase:php7.4
    
    # other docker stuff here...
    
    # expose the webserver
    EXPOSE 8080 443

    This meant, that when we merged code from development into production, we’d get an access error to the base image, if forget to change the url.

    This caused a lot of confusion, when deploying, if the developers were not used to our setup.

    To handle this, I changed our workfile to pass an argument with the base image container registry url, so Dockerfile will not be changed all the time.

    name: Build and deploy Develop
    
    on:
      push:
        branches: [ develop ]
      pull_request:
        branches: [ develop ]
    
      workflow_dispatch:
    
    
    jobs:
      build:
        name: build and deploy develop
        runs-on: ubuntu-latest
        environment: develop
    
        steps:
          # azure login steps here...
    
          - name: 'Build and push image'
            uses: azure/docker-login@v1
          - run: |
              docker compose build --build-arg CRPATH=uniquedevrepo.azurecr.io
              # other actions to run here... :)

    Notice the “–build-arg CRPATH=uniquedevrepo.azurecr.io” which is passed to Dockerfile. (skipping docker-compose.yml). We can then simply use the ARG command to get the argument and use it:

    # tell Dockerfile that we are going to use the passed argument CRPATH
    ARG CRPATH
    
    # Base path for image - using the CRPATH argument.
    FROM ${CRPATH}/projectbase:php7.4
    
    # other docker stuff here...
    
    # expose the webserver
    EXPOSE 8080 443
    

    This means that I only have to maintain my github actions workflow for each environment and no longer worry about the Dockerfile having the wrong container repositroy path for our base image.

  • Handle replication errors on Azure Database for MySQL flexible server

    At work we have just recently started moving our data to Azure Cloud. We still have the old on-premise setup running, with databases and web servers.

    To make the move to Azure Cloud, I have used a lot of time prepping the terraform scripts to handle our servers, but I have only recently been able to make a copy of the current database (with a whopping 1.7TB of data) and copy it to the cloud.

    It took a few days to insert the data into Azure MySQL, so after getting a replication error today, I was not ready to drop the replica database, to insert a fresh dump.

    Calling the following select, I could read the error:

     select * from performance_schema.replication_applier_status_by_worker;

    It was a create table error… The master database had created a new table using MyISAM, which Azure DB does not allow.

    Normally, you would increase the global counter, using:

    SET GLOBAL sql_slave_skip_counter = N;

    Azure does not support this, as it requires SUPER privileges (whatever that means). Thankfully Azure have implemented a set of methods to call, where this one exists: (remember to stop the replication first)

    CALL mysql.az_replication_skip_counter;

    This increases the global counter by 1. Then i took the created table, changed the statement to use innodb instead and ran it on the slave. Then i called:

    CALL mysql.az_replication_start;

    And the slave skipped the table create statement from the master, and continued with the replication.

    Link to Azure docs: https://docs.microsoft.com/en-us/azure/mysql/single-server/reference-stored-procedures