Popular Tags:

Skype for Business CCE 1.4.1 fails instance deployment with Access Denied error

28/11/2016 at 11:34

Last week I was deploying a new set of Skype4b Cloud Connector Edition instances for a customer. Although the boxes had been prepared a while ago using CCE version 1.3.8, I deleted all the components and went for the latest one, version 1.4.1.

No matter how many times I tried to deploy CCE, the process would fail during the clean-up steps at the end, always throwing an Access Denied error when attempting to connect via Powershell to the management interface of the Mediation Server. This would result in an unfinished deployment, the instance wouldn’t be marked as finalised in Office 365, and you’d be basically left with an unusable mess.

CCE Access Denied Error during clean-up

CCE Access Denied Error during clean-up

After pulling my hair for hours and hours (also because every attempt to redeploy would result in another 2 hours spent waiting for the VMs to be created), I managed to find the issue.

The CCE 1.4.1 scripts probably contain a bug (didn’t really investigate them as I ran out of time), where they used the wrong set of credentials to connect to the mediation server via Powershell. The issue seems to happen every time you provide different credentials for VM Admin and for Domain Admin when running Register-CcAppliance.

How to fix this

The procedure to redeploy the instance is quite simple (once you’ve figured out where all the CCE bits end up).

The first step is to uninstall the virtual machines. Because the instance is not live, running the cmdlet without arguments won’t work, so run it this way:

Now that the virtual machines have been removed, delete the Management Switch and the temp vSwitches with a GUID name from Hyper-V:

Remove redundant Hyper-V vSwitches

Remove redundant Hyper-V vSwitches

Next, remove the CceService account from Computer Management:

Delete CceService Account

Delete CceService Account

Finally, remove the credential files from C:\ProgramData\CloudConnector, they’re two XML files which contain all the credentials used by the deployment scripts. Removing them will force the scripts to prompt you again for passwords:

Delete CCE Credential Files

Delete CCE Credential Files

Now that we’re back into a clean state, let’s re-register the appliance, this time making sure that VM Admin and Domain Admin passwords are matching, and redeploy as normal.

Please note that once the deployment completes successfully, in some cases you may still need to go and manually fix the vSwitches connected to your virtual machines one by one, as CCE 1.4.1 seems to have (another) bug where it leaves the VMs connected to the temporary private vSwitches at the end of the deployment. In my experience it looks like running Publish-CcAppliance fixes them for you, but the official documentation tells you to only run that cmdlet when deploying more than one appliance. It doesn’t seem to hurt to run it on single appliances either though.

Basically, you want the Corpnet adapters to be connected to your SfB CCE Corpnet Switch, and the Internet adapters to be connected to your SfB CCE Internet Switch. Reboot the AD controller once you do the change, so that the DNS server can bind itself to the correct IP.

Lync AutoOps v1.5.2 – Now with support for multi paired pools!

12/01/2015 at 17:28

UPDATE 22/09/2016 – Minor bug fixes.

UPDATE 20/01/2015 – Just reissued a small bug fix and brought the version to v.1.5.1, as there was an array issue when dealing with simple mode operation.

Not much time has passed since the first public release of Lync AutoOps v1.1.0 and I’ve already pushed out an update, following some suggestions I received by fellow consultants.

The most requested feature was to add support for multiple Lync Paired Pools, as this would cover scenarios such as:

  • Lync Pool with multiple SBAs attached
  • Multiple global Lync Pools, both single and in pairs

Now version 1.5.0 covers the scenarios above and adds a few nice options to make life easier when dealing with complex deployments:

  • Users are now enabled based on their location (which can be any AD attribute, by default I used physicalDeliveryOfficeName, i.e. the Office attribute, but it can be customised in the config file)
  • You can choose between the old ‘simple’ method to enable users on a single pool or single paired pool, or the new MultiPool method which uses user location (and also supports single/paired pools)
  • If a user’s location is not set, or contains typos, you can set one of the pools as IsFallBackPool, and will be used when no better option is available
  • You can choose between having the fallback pool option enabled or not. If not, the above non-specific users will not be enabled and an error will be logged, so they can be fixed by hand before enabling them
  • Each pool can be associated with any number of locations, so you could have a pool in London that will enable all users for Manchester and Cardiff as well

I believe the above changes should cover most scenarios administrators face when activating users for Lync.

Version 1.5.0 also brings a couple of changes and squashes a few minor bugs, and includes a full Word doc with the documentation, since the configuration is starting to become a bit extensive.

For further guidance on installation and configuration, including how to set up unattended tasks etc., please refer to the included Admin Guide, or see my previous blog post.

Important Notes

The script is pretty intuitive, but there are two things to keep in mind:

  • Whenever you change the pool configuration for LyncAutoOps, remember to delete the LastUsedPools.config file. This file will be regenerated automatically at the next run, and it’s used to store which pools were used in the last run, so that the script always enables users evenly amongst pools.
  • You can only have one IsFallBackPool in the configuration to be used when users’ locations are not specified or they contain typos.

Sample LyncAutoOps Topologies

In this section I want to show some of the possible Lync topologies which are supported to enable users automatically:

Single Lync Pool

Single Lync Pool

For this simple scenario, we’ll set the script’s PoolTopology to Simple and just enter the first FQDN in FirstPoolFQDN, Location and IsFallBackPool can be left in place but won’t really be used during script execution.

 Paired Lync Pool

Single Paired Pool

Again, a very simple scenario. The only difference from above is that we add also a SecondPoolFQDN so that users will be enabled 50/50 on both pools as it happened in the old version of LyncAutoOps.

If for some reason you only want to keep users on the first pool, while the second stays empty in hot-standby, then use the configuration from the ‘Single Lync Pool’ paragraph and don’t enter a second pool FQDN.

 Multiple SBA Topology

Multiple SBA Topology

In this particular scenario, we have a single pool in London, with 4 SBAs attached to it. In such a scenario, we’ll just set up five pool entries, each one with their location field set up. We’ll set the main London pool as IsFallBackPool so that users without an explicit location will be homed there by default. If one of the sites is supposed to home other locations as well, simply add another Location line to the pool block. If the London pool was paired, simply add the SecondPoolFQDN and the script will enable people 50/50 on it.

 Multiple Global Pool Pairs

Multiple Pool Pairs

 Download LyncAutoOps 1.5.2

Lync AutoOps 1.5.2

Lync AutoOps 1.5.2 Admin Guide

 

Lync AutoOps – Automate the Lync User Lifecycle

05/01/2015 at 15:54

This is a topic I found myself having to face during many deployments: how will the customer’s IT department eventually manage joiners/movers/leavers in Lync once we finish the deployment?

When dealing with very big deployments, this is usually not an issue, as some kind of IDAM (IDentity and Access Management) system will be already present (such as Forefront Identity Management, or Dell Identity Manager, etc.). If that’s the case, all I do is normally provide the IDAM team with the one-liners to manage Lync users and let them know at which point of their provisioning workflows they’ll fit best.

The problem instead arises for pretty much any company below 5,000 users in my experience, as they’ll normally provision AD groups and Exchange mailboxes using independent PowerShell scripts (or *gasp* by hand every time!).

LyncAutoOps aims at solving this issue for all these companies without a proper IDAM system.

The script originated as a bunch of individual scripts tied up together with a batch file, not pretty at all, which I had to develop from scratch for a particular customer. Upon noticing I had to use and re-use them for so many deployments, I decided to tidy things up and make a unified script, reusing functions etc. I also moved all the configuration parameters in an easy to read (kind of, if you’re into XML) config file, which makes the job of implementing the script extremely quick.

What does the script do?

Essentially the script will perform 4 functions, which are the basics of Lync user management:

  • Enable Lync users who are members of an Active Directory group
    • It supports either single or paired pool. If using a paired pool set up, the script will enable users on 50/50 split between the two pools and write to a file the last pool used, so that at the next script run, the other pool will be used.
    • After enabling the users, it will assign a number of Lync policies to all the users.
    • It will also remove the users from the AD group used for activations, so the group stays empty after usage.
  • Suspend Lync users who are enabled in Lync but disabled in Active Directory. I saw many customers not realising that simply disabling someone in Active Directory doesn’t prevent them from using Lync at all, because Lync issues a client certificate that lasts 180 days. Therefore, after you disable a user in Active Directory, chances are he will be able to still use Lync on a previously-used device for another 6 months. Hardly security-conscious, is it?
  • Reactivate Lync users who are disabled in Lync but enabled in Active Directory. This does the opposite: an employee may have gone in maternity for a few months, the IT department disables her account, then she comes back and gets it re-enabled. The script will re-enable these people automatically.
  • Delete Lync users who are disabled in Active Directory and who haven’t logged in for a custom number of days (usually 90 is a good number). This allows the Lync team to keep the backend database tidy by removing all the users who have left the company. It’s always good practice to delete people from Lync before deleting the AD object, or a lot of ‘leftovers’ will remain in the Lync backend database.

On top of all this, it also emails a custom list of recipients the results of each script run. If there were errors, a separate log file containing only the errors will also be produced to simplify remediation.

Infrastructure Prerequisites

Even if I could have saved myself a fair bit of work by using the Active Directory Powershell module, I decided to use Powershell 1.0-style .NET objects to perform the AD-related operations. The reason for this is that in quite a lot of environments I’ve worked in, they either had Windows Server 2003 or 2008 R1 for domain controllers, or they had 2008 R2 but they disabled the AD Powershell. For one reason or another, as much as I’d wish all customers to migrate their DCs OS and forest functional level to the latest and greatest on the day it comes out, this situation keeps on coming up, so I decided to avoid the problem entirely and not rely on AD Powershell.

Another possible workaround I was thinking of was to use the good old Quest ActiveRoles Management Shell module, but I didn’t for two reasons: first, some companies don’t really want to install 3rd party modules onto their servers without a really good reason, and second, Dell just decided around the end of November 2014 to put the module (which was free under Quest since forever) behind a paywall for ActiveRoles server customers. Thanks Dell :-/

As outlined above, the only requirements for the script to run are:

  • Microsoft Lync Server 2013 Core Components installed on the server on which LyncAutoOps will run
  • PowerShell 3.0 or above (it should work with PowerShell 2.0 as well but I haven’t tested it)
  • PowerShell signing set to Unrestricted. I’m planning to sign the script at some point though.
  • A user with enough rights to operate on AD and on Lync users (normally CSUserAdministrator would be enough, but for Powershell operations I found out more rights are required, so I tend to give CSAdministrator rights to the service account which will run the script)
  • A new AD group in the domain which contains the users to enable, it can be called like in the script: ‘NewLyncUsers‘, but it doesn’t really matter. This is the group that the Operations team will have to remember to add new users to.
  • Windows 2003/2008/2008 R2/2012/2012 R2 Active Directory
  • All the users to enable should already possess an email address. This part is important – if they don’t have an email address already provisioned, the activation will fail, because we’re using the email as a base to compose their sip address. Should this be your situation, you will have to modify the script where it enables users, and use another criteria to generate their sip address. Please read the documentation for Enable-CsUser, especially around the -SipAddressType parameter, in order to find a viable alternative for your situation. Perhaps using the UPN? 🙂

Installation Instructions

In order to install the script easily, I recommend to unpack the contents of the ZIP file (you’ll find it at the end of the article) in the C:\ root on a server with Lync 2013 Core Components already installed. Normally, one of the Lync Server 2013 frontends will be a perfect candidate.

Configuration Parameters

In the folder, there is a file called LyncAutoOps.config, which contains all the configuration parameters required by the script. The file is commented, so it shouldn’t be difficult to fill in all the parameters for the environment in use:

Here’s a list of all the parameters, with their explanation:

  • ADSettings
    • NewLyncGroup: sAMAccountName of the group containing new users to enable
    • NetBiosDomain: Netbios domain name containing the users to enable
    • DNDomain: Search root domain expressed in distinguished name format
  • LyncSettings
    • PairedPool: Setting this value to True will cause users to be enabled 50/50 on both pools
    • FirstLyncPool: FQDN of the first Lync Pool
    • SecondLyncPool: FQDN of the second Lync Pool
  • ScriptFunctions
    • Enablement: if set to true enables users for Lync
    • Suspension: if set to true suspends users for Lync when disabled in AD
    • Reactivation: if set to true reactivates users for Lync when enabled in AD
    • Deletion: if set to true deletes users from Lync when disabled in AD and beyond the last logon threshold
    • DeleteThreshold: number of days since last logon for deletion
  • Notifications
    • EnableNotifications: if set to true it emails the recipients contained in Recipients.txt with the results of the script run
    • FromAddress: email address to use when sending out notifications
    • SMTPServer: SMTP server FQDN, normally a local Microsoft Exchange
    • RecipientsPath: path to the list of recipients to be used
  • UserPolicies
    • ClientPolicy: name of the client policy to assign
    • ConferencingPolicy: name of the conferencing policy to assign
    • ExternalPolicy: name of the external access policy to assign
    • PinPolicy: name of the PIN policy to assign
    • ArchivingPolicy: name of the archiving policy to assign
    • ExchangeArchiving: when set to true it sets new users to archive IMs in Exchange 2013 mailboxes

Recipients List

If you choose to receive the script notifications via email, you should modify the file Recipients.txt, and add all the email addresses of the people who will receive the notifications, one after another on separate lines.

Testing the script

In order to test out the script functionality I’d recommend to first try it in a representative lab environment. I never experienced major issues with it, but it’s always better to be safe rather than sorry. Before you go ahead, remember you can’t consider me liable for destroying your environment, should something fun happen there 🙂

  • Add a bunch of new users to the newly created AD group for new Lync users
  • Make sure all the configuration parameters are set correctly
  • Launch the script
  • Check that the users have been correctly enabled and that the specified user policies have been granted
  • Now try to disable some of these users in AD, and relaunch the script
  • They should be disabled
  • Do the reverse to reactivate them
  • Try to set the DeleteThreshold parameter to 1, deactivate some users, then launch the script the day after in order to delete them from Lync

Running LyncAutoOps unattended

Normally, once set up you’ll want this script to run without having to constantly babysit it, and that’s how it was designed in first place.

The easiest (and most normal way) to run something in a Windows environment is to take advantage of Windows Server’s task scheduler.

We’ll see here how to correctly create the task and make it run twice a day, every 12 hours.

First thing, start Task Scheduler and create a new task:

Create New Task

Create New Task

Next, in the General Tab, set the following parameters:

  • Name (obviously)
  • Choose the service account that will run the script
  • Tick ‘Run whether user is logged on or not’
  • Tick ‘Run with highest privileges’ (it may or may not be required)
  • Configure for: Windows Server 2012 R2 (or whatever highest level you’ve got available according to the version of Windows Server you’re running)
General Task Settings

General Task Settings

 

Next up, in the Triggers Tab, select the ‘New…’ button:

Add Trigger

Add Trigger

Configure the first trigger as follows:

  • Run Daily
  • Start it on the same day at 11:00:00 PM
  • Recur every 1 days
  • Stop task if it runs longer than 4 hours (you never know…)

Then add another task identical to the below, but set the start to the day after at 11AM (in this case that’d be 2/5/2015 at 11:00:00 AM)

Trigger Settings

Trigger Settings

Here’s what the trigger window will look like once you’re done:

Triggers set up

Triggers set up

Next click on the Actions tab and create a new action. This step is very important. Set it as follows:

  • Action: Start a program
  • Program/script: Powershell
  • Add arguments: -command “& {.\LyncAutoOps.ps1}”
  • Start in: C:\LyncAutoOps

The ‘start in’ folder is very important, as all the config files and logs will be created relatively to this folder.

Set up of the action

Set up of the action

Now you can confirm and save the entire task. The Task Scheduler will prompt you for the service account password now, as it will need it to run your task.

Screen Shot 2015-01-05 at 15.19.22

 

That’s it!

To test if the script works, the quickest way is to disable a test Lync user in AD, then click Run manually, and check the Logs folder, to see whether a new log file was created.

Hope you enjoyed this article. Let me know if you have any questions or new features to integrate and I’ll try to accommodate them.

Downloads

LyncAutoOps-v.1.1.0.zip

A very simple AD Watchdog to monitor user changes

31/10/2013 at 15:32

At some point, I needed to create this script due to an overzealous identity management system installed at a client. This system was, more often than not, stripping away vital attributes from several users, such as the Exchange UM proxy addresses, and remastering the attributes without the values.

Because of this, the support team would often find itself dealing with tickets reactively, especially if they were raised by VIP users, and the AD logging tool was just overwhelming to look at due to the sheer size of this environment and the amount of changes taking place every second. This script solved the issue, by alerting them as soon as something would change in Active Directory for a list of selected users.

How it works

The script looks for a file called MonitoredUsers.txt which should contain all the sAMAccountNames for every account you need to monitor for changes, one every line (example attached in the ZIP file).

It then connects to AD, retrieves the object for the user, and if it didn’t exist already, it will export an XML representation of the object into the database subfolder. This becomes the baseline.

At the next run of the script (it should be scheduled really), if a user already exists in the database folder, this will be compared to the current AD records. If they match, nothing is done. If they don’t match, the log will contain the differences from before, and the current records will become the new baseline in the database.

At the end of the run, the log file will be emailed to a list of recipients contained within Recipients.txt (one address on each line).

It wouldn’t be too hard to tweak the script to do the opposite and monitor a specific attribute for the entirety of Active Directory if desired.

Before using it, make sure to replace the $Smtp variable with your actual SMTP server and add some users to the MonitoredUsers.txt file (if using Exchange you need to be running this script with enough permissions to send emails via Powershell with a spoofed address).

Also, after running it for a while, I noticed that some attributes would just constantly change and would increase the noise in the log. For this purpose, I added a section in the middle of the script where you can define a series of attributes to skip.

Download: ADWatchDog_v1.0.zip

Easily retrieve the patch level of a Lync 2010/2013 deployment

30/10/2013 at 16:27

As a consultant, sometimes I get asked by clients to retrieve what the patch level of their deployment is. Most of you know what a tedious process is to gather this information, due to the involvement of WMI, cumbersome revision numbers I never remember and scattered information about the cumulative updates released for Lync.

Hence why I wrote this little script, which allows you to do just that, by running it against your deployment, it’ll return what version of the core components is running on each server and specify the CU number:

This is the resulting table after running the script on a Lync deployment.

This is the resulting table after running the script on a Lync deployment.

Please note that I didn’t have a chance to test it on Lync 2013, so a few of the new roles may fail to be retrieved, but generally it should work just as well.

I added a few settings at the top of the script to help tailor the collection of servers to your specific environment (marked in yellow).

UPDATE 02/05/2014: Added CU11 and CU12 for Lync Server 2010 and CU4 for Lync Server 2013 to the list.

You can download the script here:

Download: Get-CsServerVersion.ps1_v1.1