I’ve got a bunch of pet websites, some hosted in Azure, others with an old school hosting provider. I don’t think these other non-Azure sites are going to be transferred any time soon either by virtue of the fact you typically get an email server with your web hosting package, something sadly lacking from Azure.

Most of the source code for these site is in GitLab. Now, I like both Azure DevOps and GitHub but at the time these sites were created both of these cost money for private repos and despite that changing recently I don’t really see a compelling argument to shift the code.

What I don’t have for some of the sites (and would be useful) is to set up continuous integration and/or deployment to build, publish and deploy them.

I’ve split this post into two parts. This first part will describe the situation where the source code is in GitLab, the website hosted with a shared hosting provider and the deployment process itself is handled using Azure DevOps.

GitLab Setup

Not much to do here, all you need is an Access Token. To get one log in to your GitLab.com repo and go to your User Settings. In the left hand menu select Access Tokens.

Create GitLab Access Token

Give the access token a name and then select the following permissions:

  • api
  • read_user
  • read_repository
  • read_registry

Click the “Create personal access token” button. It’s important to take a note of the token’s value here as it’s NOT shown again when you come back to this page. If you do lose it then you can revoke the existing token and generate a new one.

That’s all you need to do within GitLab.

Hosting Provider Setup

Depending on your hosting provider you may have a few options for deploying a website. In my case I’ve got the choice of Microsoft’s Web Deploy or good old FTP.

I’ve used Web Deploy in the past and it works well when you’re doing a quick and dirty deployment straight from Visual Studio to the web server. I’ve got the slight complication of having CloudFlare sitting in front of the website which is great for performance and also gives you the option of HTTPS for free even when your hosting provider may not support that. The reason it’s complicated is that Web Deploy operates over port 8172 by default and this isn’t a port that CloudFlare forwards traffic for. So traffic for, let’s say, https://www.example.com would be forwarded (on port 443) butt traffic for https://www.example.com:8172/msdeploy.axd?site=example.com wouldn’t.

Now it might be possible to talk nicely to your hosting provider and ask them to offer the Web Deploy service on either a different port or alternatively on a sub-domain – and then ensure that CloudFlare doesn’t proxy traffic for this specific sub-domain. Neither of these were an option for me so we’re left with FTP. All you need from your hosting provider are some credentials for connecting to your website via FTP.

Azure DevOps Setup

Service Connections

Now that we’ve got a way of connecting to both our source control provider and our hosting provider we can configure DevOps to pull the source code, build and publish, then copy the published files to the website’s location.

Log in to your Azure DevOps organisation site. If you don’t have one you can sign up for free. Open (or create) your project and then open the Project Settings page – the link is in the bottom left of the page at time of writing.

On the Service connections screen click New service connection and select Other Git.

Other Git Service Connection

Click Next.
Enter the URL to your GitLab repository. In the Authentication section paste the access token you saved from earlier into the Password/Token field. The username isn’t required.
Give the connection a name and Save.

Git Service Connection Details

Now we want to create another connection, this time to the FTP site. Click New service connection again, this time choosing Generic.

Generic Service Connection

Enter the URL of your FTP site, the credentials required to connect, give the connection a name and hit Save.

Generic Service Connection Details

Now on your Service connections screen you should see both connections.

Service Connections

Now we’ve got our service connections configured we can set up our pipelines – one for the build/test/publish process and one for the release process.

Build Pipeline

Open your Pipelines screen in DevOps and click the New pipeline button.
Select the Other Git option.

New Pipeline Connection

On the following screen the default options should be correct. The source should be Other Git and DevOps should have picked up your previously created GitLab connection. Ensure the correct branch name and click Continue.

New Pipeline Repository

Next, we need to select the template for the pipeline. As I’m working with a .NET Core web app I’m selecting the ASP.NET Core template. Hover over your chosen template and then click the Apply button.

New Pipeline Template

Here’s what that template (at time of writing) looks like.

New Pipeline ASP.NET Core Template

I’ve given it a name of CI Build. Feel free to configure the rest of the pipeline and job steps as necessary for your project. The default setup works just fine for this project.
The final step in the process, Publish Artifact, picks up the published files from the previous Publish step and copies them to a drop folder that we can access in the release pipeline.

Click Save & queue to save your pipeline and kick off a build.

While that’s running open the Releases page in DevOps and click New release pipeline. On the template selection screen click the Empty job link at the top.

New Release Pipeline Template

In the resulting screen you can change the name of the stage if you wish. Stage 1 works for me though.

New Release Pipeline Stage

On the canvas click the + Add link next to Artifacts.

Your project should be pre-selected. Choose the build pipeline we’ve just created and feel free to change the source alias.

New Release Pipeline Add Artifact

Click the Add button.

Now back on the canvas, hover over the click the “1 job, 0 task” link under Stage 1.

New Release Pipeline Stage Tasks

On the stage screen click the + button in the Agent job box.

New Release Pipeline Add FTP Upload Task

Find the FTP Upload task and click the Add button to add to the stage tasks.

The task will be added to the list – click on it to configure.

FTP Upload Task

There’s a few things to set here:

  • The name if you want to change it.
  • Select the FTP service connection created earlier in the FTP Server Connection drop down.
  • The Root Folder should be set to the drop folder or a folder within that contains the files you want to copy to the FTP site.
  • Update the File patterns if necessary – I’m just copying everything.
  • Update the Remote directory if your deploying into a sub-folder on the FTP site.
  • Set the advanced options as required – I’ve set Preserve file paths as otherwise the folder structure gets flattened on copying.

Once your happy with the settings click the Save button in the top toolbar, choosing the folder where you want to save it (default root folder works in most cases) and a comment.

Now that you’ve done that you can perform the deployment by clicking the Create release button in the toolbar and then the Create button in the resulting popup.

You can click the Release-1 link at the top or navigate in back through the Releases screen to view the progress.

Release Succeeded

To view logs of the full process, click on the Succeeded link in the Stage 1 box.

Release Succeeded Logs

Job done!

You’ve successfully pulled your source code from GitLab into Azure DevOps, built and published it then copied the published site to your FTP server.

Now given that this is just plain old FTP there’s a chance that if the site has a fair amount of traffic some of the files might be in use and therefore the upload may fail. There’s probably a few ways to get round this. Depending on your hosting company you may be able to connect to an API to take the site offline. Alternatively you could have a multi-stage release pipeline that uploads the app_offline.html file to the server first, then performs the copy and finally deleted the app_offline.html file.

I’ve not tried any of these approaches – I’ve only seen errors a couple of times so far and I’m happy enough to re-deploy manually when I get the email from Azure DevOps telling me the pipeline has failed.

In Part 2 I plan to do something similar where the source code is also in GitLab but I make use of the GitLab.com CI/CD pipelines to build and subsequently deploy to an Azure App Service.


There are a few posts already out there dealing with this but I couldn’t find one that managed to cover all the steps in enough detail for me. So this post will attempt to rectify that.

What do we need?

  • A web site to test. I’ve got a pretty bare-bones ASP.NET Core web application (just using the Visual Studio 2019 template) that I’ve been playing around with recent versions of Entity Framework Core in. I’ll write the tests against the home page of that site.
  • A test project. For convenience I’ll be adding the test project to the same solution as the web application.
  • A release pipeline. For unit and integration tests you’d normally run these as a step in a build pipeline, preventing the deployment if the tests fail. For UI tests though I need something to be deployed in order to test it. I’ll be adding the test step to a release pipeline.

The web application

As stated, I’m using the template ASP.NET Core web application in Visual Studio 2019. Here’s the project structure. I’ve added some data context related stuff but you don’t need any of that for the purpose of these tests.

Visual Studio project structure

Debug this project and you should see something resembling the following.

Web application home page

And that’s it. Now I need to publish this somewhere. I’m lucky enough to have an Azure subscription at my disposal currently so that’s where this will live. Of course the site can live anywhere publicly accessible.

Let’s move on to writing some tests for this web application.

The test project

First, I’ll create a new project in our existing solution. Right click on the solution in the Solution Explorer pane and select Add > New Project….

In the first dialogue select Class Library (.NET Core) then click Next.

New project type

In the next dialogue give your project a name and a home. Click Create.

New project name

Visual Studio will open up the newly created Class1 class. Rename this to something sensible (I’ve chosen HomePageTests) and lets start adding the dependencies.

I’ve added the following nuget packages:

  • FluentAssertions (Version=”5.6.0″)
  • Microsoft.AspNetCore.TestHost (Version=”2.2.0″)
  • Microsoft.NET.Test.Sdk (Version=”16.1.1″)
  • MSTest.TestAdapter (Version=”2.0.0″)
  • MSTest.TestFramework (Version=”2.0.0″)
  • Selenium.Support (Version=”3.141.0″)
  • Selenium.WebDriver (Version=”3.141.0″)
  • Selenium.WebDriver.IEDriver (Version=”3.141.59″)
Visual Studio tests project

Before I write the actual test method I’m going to add some setup and tear down methods in the class. Not strictly necessary for a class with a single test method but as the test project grows it’s useful to have this stuff in a single place.

First up, the using statements – I hate it when you see code posted for a class or method without these included.

using FluentAssertions;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.IE;
using System;
using System.IO;

Next, decorate the class with the [TestClass] attribute.

namespace EfCoreSandbox.Tests
    public class HomePageTests

Now I can add some variables. The webAppBaseUrl is the URL for the publicly accessible website. You wouldn’t normally hard-code this but rather have it in some kind of configuration store but this will do for now.

private const string webAppBaseUrl = "https://efcoresandbox.azurewebsites.net/";

The IWebDriver is the interface that all the Selenium web drivers (the things that open up and actually drive the browsers) implement.

private static IWebDriver driver;

Next up, ClassInitialise. This runs, provided it’s been decorated with the [ClassInitialize] attribute, once before running the tests of the class. In this method I call a method to set up the web driver and then navigate to the web application’s URL.

public static void ClassInitialise(TestContext testContext)
    driver.Url = webAppBaseUrl;

Then, on the flip side, the ClassCleanup method. As you’d imagine this runs once and after all the other test methods have been run.

public static void ClassCleanup()

Here’s the SetupDriver method called in the ClassInitialise method. I won’t go in to too much detail here as this isn’t a post about Selenium testing but this method configures some options for the IEWebDriver (if you have to test IE automate it – otherwise you’ll need to use IE and nobody wants that). I get the path to the web driver executable – this differs between the local machine and the Azure platform, environment variable on Azure, local directory on the local PC. Lastly set the driver variable to a new InternetExplorerDriver. If any of this fails call the TeardownDriver method to clean up.

private static void SetupDriver()
        InternetExplorerOptions ieOptions = new InternetExplorerOptions
            EnableNativeEvents = false,
            UnhandledPromptBehavior = UnhandledPromptBehavior.Accept,
            EnablePersistentHover = true,
            IntroduceInstabilityByIgnoringProtectedModeSettings = true,
            IgnoreZoomLevel = true,
            EnsureCleanSession = true,

        // Attempt to read the IEWebDriver environment variable that exists on the Azure
        // platform and then fall back to the local directory.
        string ieWebDriverPath = Environment.GetEnvironmentVariable("IEWebDriver");
        if (string.IsNullOrEmpty(ieWebDriverPath))
            ieWebDriverPath = Path.GetDirectoryName(AppDomain.CurrentDomain.BaseDirectory);

        driver = new InternetExplorerDriver(ieWebDriverPath, ieOptions)
            Url = webAppBaseUrl
    catch (Exception ex)
        throw new ApplicationException("Could not setup IWebDriver.", ex);

And the TeardownDriver method. Pretty simple – just clean up the resources of the driver.

private static void TeardownDriver()
    if (driver != null)
        driver = null;

Time to write a test. I’m keeping it simple and am just going to test that the home page has an <h1> heading tag that contains the string “Welcome”.

public void HomePageHeadingContainsWelcome()
    // Arrange/Act/Assert

And that’s it, put all this together and you should be able to run the test. If you don’t already have the Test Explorer window open in Visual Studio open it from Test > Windows > Test Explorer.

Visual Studio Test Explorer

In the Test Explorer pane click the Run All button – left most button in the toolbar. The test engine will chug away for a bit opening Internet Explorer and loading the web application and all else being well the test will pass.

Visual Studio Test Explorer Passed Tests

Next, let’ set up the release pipeline to run these automatically.

The release pipeline

While the UI tests will run as part of a release pipeline, that pipeline will pick up and deploy the output of a previous build pipeline. So lets get that set up first of all.

In your Azure DevOps portal, open the project and then select Pipelines in the left hand menu.


Click the Create Pipeline button. I’m not yet a fan of YAML so click the “Use the classic editor” link.

Choose the correct settings for your source repository, for me this is an Azure Repos Git repository. Team project and Repository are both EF Core Sandbox and I’m basing it on the master branch.

New Pipeline

Click on the Continue button. I’ve chosen the built-in ASP.NET Core template but feel free to choose something more appropriate to your needs.

ASP.NET Core template

Click Apply to continue. Now I need to do some tweaking to the pipeline.

At the pipeline level I’ve renamed the pipeline to EF Core Sandbox CI and changes the Agent Specification to windows-2019. Also, given we have no unit tests, I’ve cleared the Project(s) to test field and removed the Test step in the pipeline. The pipeline should look like this for the moment.

Initial build pipeline

Now I’m going to split the existing Publish task in two. Currently it will publish both the web applications and test projects as zip files. That’s not going to work for the test project so split them up. I’ve renamed the Publish task as Publish Web App and added a new Publish Tests task, configured as follows.

Publish Tests task

Importantly, uncheck the Zip Published Projects and Publish Web Projects checkboxes then set the Path to Project(s) field as the path to your tests project only. Mine is set to “**/EfCoreSandbox.Tests.csproj“.

Set the Arguments field to “–configuration $(BuildConfiguration) –output $(build.artifactstagingdirectory)“.

Click Save & queue to run the build. After a while your Pipelines screen should resemble the following.

Pipeline success

The last major part of this process is to set up the release pipeline. Click on Releases in the left hand menu and then on the New pipeline button. In the “Select a template” panel click the “Empty job” link.

I’ve renamed the stage to Deploy web app and closed the Stage panel.

New release pipeline

In the Artifacts panel click + Add. Choose Build as the source type and then select the appropriate project and build pipeline previously configured. Click the Add button.

Add build artifact

In the Stage panel click the Deploy web app stage to configure it. Under Agent job click the + (plus) button and add an Azure App Service deploy task. Set this up to publish to your Azure app service. I won’t go into too much detail here as your configuration will inevitably differ from mine.

Azure App Service deploy task

Click the + (plus) button again and this time add the “Visual Studio Test Platform Installer” task. This task should need no additional configuration.

Visual Studio Test Platform Installer task

Click the + (plus) button again and the last task to add is a “Visual Studio Test” task. For the Test files field I’ve specifically chosen the EfCoreSandbox.Tests.dll to run and I’ve specified that the Text mix contains UI tests. For the Search folder I need to be more specific with the location – changing it to $(System.DefaultWorkingDirectory)/_EF Core Sandbox CI/drop

Visual Studio Test task

Give the release pipeline a name (I’ve chosen EF Core Sandbox Release) and save it.

In the real world I’d consider running this pipeline in response to a trigger but for now just click Create release in the toolbar (top right) and then on the Create button.

Release created

Click on the Release-1 link that appears at the top of the screen. Mine is Release-3 because I’ve been mucking about with the pipelines, generally getting things wrong, and have created a few more test releases.

Deploy Succeeded

Hover over the Deploy web app box and then click on the Logs button when it is shown. In the Logs screen click the VsTest – testAssemblies item in the list. Scroll down a bit and you should see that the HomePageHeadingContainsWelcome did indeed pass.

Passed UI Test

So there you have it, Selenium UI tests running as part of your deployment pipeline.


This is all great, but, what about when the tests fail. Let’s face it, if you knew all the tests were going to pass every time would you bother writing them?

Let make the test fail. In the web application project change the content of the <h1> element. I’ve opted for “EF Core Sandbox“. If you’ve got your UI tests project configured to be able to run against the local web application you can run the UI tests locally to confirm that they fail.

Next, I’ll update the HomePageTests class to provide me with some additional information when a test fails. In the class add the following variable declaration.

private static TestContext testContextInstance;

Now update the ClassInitialise method to set the variable.

public static void ClassInitialise(TestContext testContext)
    testContextInstance = testContext;
    driver.Url = webAppBaseUrl;

And add the following method to the class.

private static void TakeScreenshot(string fileName)
    Screenshot ss = ((ITakesScreenshot)driver).GetScreenshot();
    string path = Path.Combine(Directory.GetCurrentDirectory(), fileName);


Finally, for the test class, add the following TestCleanup method that will be executed after every test method.

public void TestCleanup()
    if (testContextInstance.CurrentTestOutcome != UnitTestOutcome.Passed)

As you’ve probably guessed these code additions will take a screenshot of the web page if the previous test has failed and then add the file as a result file to the test context instance.

Go ahead and push the changes into source control then kick off a new build – unless your build pipeline is already CI triggered. Once the build has run (and succeeded) create a new release. Once the release run has completed you should see something slightly different, if not unexpected.

Failed deployment

Drill down into the logs as before and you’ll be able to see what went wrong.

Failed test logs

As you’d expect we can see that the test failed with the message: Expected string “EF Core Sandbox” to contain “Welcome”.

What about the screenshot though. If you click Tests Plans in the left hand menu and then Runs you’ll see the list of completed test runs – the latest failed one should be at the top.

Tests runs

Double-click (yep, I know) on the latest run with the warning icon to see the run summary. From there click the Test results link just above the toolbar and under the Run number. This will list all the tests that have failed. Since we only have one double-click the HomePageHeadingContainsWelcome test. Here you’ll get the error message and stack trace for the failed test along with, about halfway down the page, an Attachments section that should have one file.

Test result

Clicking on the attachment name will download the file. Open the download to view a screenshot of the web page at the point the test failed.

Web application screenshot

That’s it. If you want the code but don’t want to copy and paste all of above sections individually here’s a link to the complete HomePageTests class. https://gist.github.com/stuartwhiteford/bc21df9e1b98785beef0a6ed66b8c4f8

Happy testing!