I updated Release Management and my builds fail: now what?

This is a quick but worth tip to remember.
If you upgrade Release Management, you always need to update the Client. Up to this, fair enough.

But you are going to start getting errors like “The account running the TFS build service needs to be added as a system user in the Release Management Server” in your build.

What you might be missing is that in order to allow communication between the two you also need to at least open the client, and if you keep getting errors you might also want to remove and re-add the Release Management URL into it.

This operation will recreate the authentication tokens and you can use VSRM again.

Things to remember when you work with Azure and the new Team Build

Azure is an amazing piece of technology – its capabilities are immense and there is always something new to learn and use for solving our problems.

It is quite a while I am spending time on it, together with the new Team Build and a few other bits and pieces all around. I thought about summarising these little issues you might face as well, so you should save some time

First of all, please remember this: if you don’t have a build task OOB, and you don’t want to write your own, you can still use PowerShell! Never forget how powerful it is, because it can be used in literally thousands of ways.

If you are working with TFS 2015 RTM you won’t have the new build tasks bundled with the Update 1 or support for Service Principals, but don’t worry – you can achieve the same results with a bit of PowerShell.

For example, bearing in mind that creating a Service Principal for your Azure Active Directory is as easy as the first part (steps 1, 2 and 3) of this blog post, this is all you need to do in a PowerShell script to use the Service Principal for authentication:

At the moment support for Service Principals is limited to Azure Resource Manager, so for deploying any components you might want you still need to rely on the old certificate-based authentication.

The issue here is that the combobox in the build task for Web App Deployment (VSTS or TFS 2015.1) doesn’t specify anything like that. What I suggest is to follow the following approach and add the Azure subscription with both authentications:

So you would get a subscription in the comboboxes anyway, regardless of the authentication method you chose.

Eventually, bear in mind that the new build agents expose capabilities – so if you want a specific agent with Azure PowerShell installed you would need to specify AzurePS as a demand in the build definition.

What’s happened to the Visual Studio installer?

Well, they sorted it out! The Visual Studio 2015 installer is now componentised:

It is now very straightforward to understand, especially for upgrades. This is true also for the third party components like the Android development stuff.

How was that possible, given the monolithic ISO that Visual Studio was? Let’s have a look at the packages:

And then there is a file for the external, downloadable components. It is the feed.xml file, this is an extract:

Kudos to the team!

Endpoints in a Team Project: what shall I do with them?

If you roam around a TFS/VSTS Team Project Administration page you will surely come across the Services tab:

What am I supposed to do with these? Why is there “Endpoints” and “XAML Build Services”?

Let’s start with the first question – in VSTS or TFS you can connect external services to interact with various features, like Build or Release Management.

Azure is a great example in that regard, because you can link a subscription to a Team Project for deployment purposes, once you do so all the combobox related to Azure Deployment (ARM, Web App, Cloud Services) will be automatically populated.

But there is not only Azure:

The Generic Service Endpoint enables third party services like Perfecto to integrate with the platform – you need to provide a URL and some authentication.

Subversion? Yes, also SVN is supported on VSTS and TFS 2015.1. It is something new – the Java ALM team delivered this integration to enable Subversion as a repository for build definition. Kudos to them!

Jenkins is for the build engine as well, so you can use it in the OOB build tasks.

Now onto the second question – the separation between XAML Build Services and other Endpoints is because the former handles services in a different way.

But as it is more and more becoming something old and deprecated, only Azure is available for these. Moreover, the XAML Build Service Azure integration doesn’t support Service Principals out of the box so you need to customise it should you use them.

Shopping in the Marketplace with Visual Studio Team Services

With the rename, Visual Studio Online Visual Studio Team Services introduces the Marketplace for its extensions.

It is a very welcome addition to the platform, as it enables scenarios of customisation in a very easy way.

Once you are logged in your VSTS account, you can browse and choose an extension:

Each extension has its own page with all the details, and it is the entrypoint for the installation:

The amazing thing is that it is all on your service – clicking Install brings you to an installation page, where you need to select the VSTS account you want to install the extension in:

That’s it! Once installed, you can go back to your account and start using it!

It is transparent for the users, which will just see the extension working as a feature of VSTS. All the extensions are managed at account (hence Collection) level.

Upgrading SonarQube to 5.2, my notes

Today I updated our production instance of SonarQube to the eagerly awaited version 5.2.

It is such an important release because it finally embraces a full client-server paradigm (whereas the runners had to talk straight to the database before of that), it introduces support for Integrated Authentication for its database, and more.

The documentation rightly suggests to not copy across the old sonar.properties file but to carefully cross-check the settings you are modifying – it is a very wise suggestion, as the strings are now different (albeit they look similar at first sight).

The JDBC driver for SQL Server is now different, it doesn’t use jTDS anymore so keep an eye on your connection string. And there is also support for SQL Azure, which is a very nice touch if you host SonarQube on it. If you want to run Integrated Security you have to fiddle a bit, downloading the SQL JDBC driver and manually copying it in the appropriate folder. Everything is documented though.

SonarSource introduced a wizard (a first, I think, in the Java world…) for the upgrade – but beware, the wizard will start only if all the connection and configuration strings are set in the right way.

All the runners scanners must be updated, pointing straight to the SonarQube instance instead of the database, and eventually another very minor issue – if you have a custom image (the top left one) in your SonarQube instance you need to reset it and set a width, as it isn’t preserved.

Small touches I like in the TFS 2015 Update 1 Upgrade Wizard

Microsoft is always working towards improving the user experience of its products, and Team Foundation Server is no exception to this rule.

There are two bits I really liked in the Upgrade Wizard of 2015.1: the new Build Agent configuration and the Upgrade Progress monitoring page.

For the first, now it is totally clear what are the paths a Build Agent (referred to the new Team Build, of course) uses:

For the second, I can now monitor not only the step of my upgrade but also the elapsed time, and if there is any problem going forward.

At the bottom there is a very handy link to the logs, and if something happens during the upgrade, the Issues link brings you straight to the relevant log:

Again, they are all small touches, but very relevant regardless of how often you use them

Dashboards in Team Foundation Server 2015 Update 1

Another great new feature in TFS 2015.1 is the introduction of a real dashboard instead of a pre-built welcome page.

This is an example:

This is an upgraded Team Project by the way. There are a few out-of-the box widgets, but is there a Definition of Done widget? Of course not

You now have the flexibility of creating your own widgets, including a markdown widget:

Everything is customisable – this is my markdown example:

You can also create multiple dashboards – this is extremely handy if you have different teams with different focus across the same Team Project:

Each dashboard can be totally different, they are independent with each other.

Eventually, there is a new Welcome page experience. This is Markdown as well:

and there is one page for each repository by default, as it is based off the readme.md file created into each repo.

This is really a very welcome addition to the platform, as it enables even more collaboration and information’s sharing scenarios as ever.

TFVC and Git in the same Team Project: a revolution

TFS 2015 Update 1 is literally hot off the press, and I am already on it as the changelog is massive!

The #1 priority to me was to have a look at Git and TFVC in the same Team Project. This is a really important milestone for the future.

It means the Version Control storage become agnostic – you can have Git repositories in existing TFVC-based Team Projects (a legacy evolution if you want), but also the other way round, a TFVC storage into a Git-based Team Project.

The experience is as easy as it, you just need to go to the _admin page and under the Version Control tab you can manage that:

If you already have TFVC configured in your Team Projects, you can’t add another repository of the same type of course:

Why should I add a TFVC repository if I already use Git? Well, migrations, large file storage are two examples.

You need to run Visual Studio 2015 Update 1 (currently in CTP1) to fully access these new repositories - which obviously highlights that you have both the Version Control systems in place.

The next thing I will look at (across the weekend ) are the dashboards!

Problem solved! Git-LFS and VSO

One of the problems you might face with Git is the performance hit when you start storing binaries into the Version Control.

It is actually by design – Git takes content snapshots at every commit, and it can’t handle deltas on binary files like on text files.

Fair enough, it has lots of other pros (and cons too), but the inability of storing binaries in an easy and non-disruptive way hampered the need for a shared tool in an heterogeneous development teams. If you have people working on the UI of your applications and you want them to version the .psd files they use, you can’t use Git.

Well, you couldn’t. GitHub developed Git Large File Storage for sorting this issue, but it wasn’t by default and eventually you would need to pay for usage (over 1GB quota of free space and bandwidth). It isn’t a out-of-the-box solution.

But yesterday Microsoft announced Git-LFS support on VSO, and it is a game changer to me.

Firstly, it is enabled by default on all the Visual Studio Online’s Git repositories. What you need to do is to install the Git-LFS extension, and nothing else.

Then, it is free for unlimited storage, so you don’t have to worry about limits, quotas and usage. It’s there, use it (if it makes sense, obviously).

Eventually, it will be included in Team Foundation Server 2015 Update 1, meaning that you will get exactly the same experience on-premise.

That’s marvellous, really. It solves the aforementioned issue in an effortless and easy way, making Git even more approachable.

Personal Access Tokens in Visual Studio Online

When you try to access some services in Visual Studio Online, you might need to enter your Alternate Credentials. Think about Git, for example.

This approach works, no questions about it. But in terms of security it isn’t the best choice. It isn’t granular at all and credentials have no expiry date.

But Visual Studio Online also provides Personal Access Tokens, to fix this inconvenience. A Personal Access Token offers better granularity and expiration management:

And how to use it? You need to safely store it (you can’t access them after the creation, by design), and then you can use the string in place of the password when asked.
The username in that case can be whatever, it is just not used.

Why shall I split the columns in the Kanban Board?

A very interesting discussion came out at the SmartDevsUG meeting last night about the Kanban Split Columns in TFS and VSO. What is the real rationale behind it in a real world project? Why should I have a Resolved column split in Doing and Done, for example?

Without taking into consideration the Kanban principles behind this tool, I have an easier explanation. If you step back from the technical stuff for a moment and you think about the business side of it, it is pretty clear.

For example, you are working on a customer hotfix for your product – the code is done, the fix is tested thoroughly and you are technically done. But you aren’t completely done – did you deliver the hotfix to the customer? Or from a different perspective, is it billed? Until this hurdle isn’t cleared, you are in a Resolved-Done situation.

Reading the TFS update logs

Did you ever read a TFS update log? If not, you should do it.

Once you do anything on the TFS databases, all the operations are logged, obviously. You can reach these logs from the Administration Console:

The amount of data in there is really helpful – for example, each Stored Procedure used by the installer is detailed, giving you an idea of what happens to your databases during each update:

And you also get a recap of how long it took with each individual timing:

That is interesting enough if everything works, but what about when something fails?

Well, that’s even better, because if you experience an error whatsoever, you will get a detailed exception stacktrace in the logs:

This gives you a real actionable insight before touching anything or raising the phone to call CSS.

Draft Builds in the new Team Build

With the new Team Build you have the possibility of working on Build Definitions in an asynchronous fashion, without impacting other team members or wasting time cloning the existing build.

For example, you might want to add some tasks to an existing Build Definition – you can then Save as a draft..

You are actually creating a clone of the existing Build Definition with the new steps you added. You can even run it!

All the builds you run with this definition are marked with a .DRAFT in the name:

This is not different compared to any other build – it is just not affecting the existing published builds:

Then once you save it everything is merged with the original definition, with no further effort.

Quickly change the TFS Integration Platform settings

You installed the TFS Integration Platform, and now you want to move your database. Or you want to change the temporary folder used during the migrations.

Shall you waste time reinstalling it?

No, what you need to do is to modify some strings in the MigrationToolServers.config file:

Pre-upgrading a large Team Project Collection to TFS 2015

TFS 2015 is out and we all rush upgrading our Team Project Collections!

But if you have a collection bigger than 1TB you should stand back and read some documentation before that. This page in particular explains what you should do if you want to upgrade such a huge collection.

The reason for using TfsPreUpgrade is very simple – you don’t want your users stuck for days while you upgrade Team Foundation Server. So the tool is going to partially upgrade the schema while the server is still available, so the actual upgrade (the one done by the installer) takes no longer than two days, which means a weekend, which also means no perceivable service interruptions.

You can run TfsPreUpgrade from whatever machine you like. In my testing environment I am using at the moment I am running it from a plain Virtual Machine which is going to be used by the Application Tier, but it has nothing installed yet. You basically need access to SQL Server, and that’s it.

First thing, run it with the Estimate switch. It will give you an idea of the time and space needed for the upgrade:

To give some context, this is data for a 2TB Collection. A few hiccups while running the Run switch might happen, especially with a temporary test environment:

but you can just relaunch TfsPreUpgrade and it would start again from the same failed point after a quick check. It is smart enough to not further expand your databases if you don’t need it:

Once it is done your offline upgrade will be much faster, because these operations were carried in advance!

Change the Release Management URL when the console is stuck

Quick one, but helpful IMHO. After updating Release Management Server or Client you might experience this:

and then you would be stuck – you can’t change the Release Management Server URL.

The quick and dirty solution would be changing the URL in the Microsoft.TeamFoundation.Release.Data.dll.config file.

A better solution is to run this command line tool, which basically automates the former. In the worst scenario, reinstall the Client and it would prompt asking for a Release Management Server URL.