Some of you are properly using the very cool feature on Azure App Services called “ Deployment slots ” which lets you swap different instance in and out of your production website 🚀. Hopefully you are also using Application Insights to gather telemetry from your app.
But you might not have found out that the name or your website instance doesn't get swap when you move between the different slots. This means that if you are swapping ex. site.azurewebsites.net with site-dev.azurewebsites.net your "production" web app is actually still named site-dev.azurewebsites.net in your telemetry.
This bug took me and my team a while to figure out and we ended up making a request to Azure support to file a case. The reason for the bug is, that behind the scenes there is some kind of IIS website with a instance/role name. That name is not changed during a swap, because a role name change requires a restart, which will result in a cold start and defeat the purpose of the swap feature.
A nasty little learning from the real world 🤔