For quite some time now, I’ve been working on a global design/deployment where there are multiple pools located throughout the globe. Throughout this design, I’ve seen the flexibility of Lync Server 2013 at its very finest and gained a much better appreciation for how Lync Server was designed by the Product Team at Microsoft. Due to the shear size of this particular design, I’ve seen Lync have to perform as advertised with not much margin for error. There has been a few hiccups, but nothing that a little forward thinking or creative thinking couldn’t solve. Now with all the praise given, I do have to say that some of my pain points along the way were not well documented by the folks at Microsoft, so I’ve had to discover a few things as I’ve pushed forward.
As we know, Lync 2013 introduced Mirroring as a replacement for Clustering in the great search for respectable Disaster Recovery. Even though the Lync team’s selection of Mirroring was a much better practice than clustering, it did spawn a whole new argument; Why Mirroring over AlwaysOn?”. Find yourself a SQL guy and have a nice debate. J Since we are using Mirroring, we as Lync Consultants must learn all the new tip and tricks that SQL Mirroring brings to the table. We now straddle that fine line of who owns the Mirror and how much SQL do we need to know to accomplish our goals. With this particular design, the SQL Team was great to work with so I got to learn quite a bit along the way. One thing I did learn, was the reason for this blog and I find it to be quite the Gotcha that TechNet failed to tell us. I’ll also call out a few more things that I’ve learned with this such deployment.
Lessons Learned from my experience:
1.) TechNet states you can use a single Witness for all your pools. This is correct, but if you have multiple central sites, that Witness needs to be defined in each “Shared Components” under each central site or it will not show up as an option.
2.) If you have deployed a Single SQL Server in the Topology then decide to come back and deploy a mirror at a later time, you cannot use “Install Databases” under your targeted Pool. You must use either PowerShell or configure the Mirror in Topology then Publish the Topology. This may have been permissions related, but nonetheless, it was the only way Mirroring after the fact worked for me. Keep it in mind.
3.) When an organization employs Role Separation and permissions in SQL are tightly monitored, it is very important that the DBAs setup the permissions appropriately between the Mirror Nodes and the witness. In my experience, we had dedicated SQL Boxes which was not an issue as we had SA permissions, however, the Witness was a “Shared” SQL Box, so the permissions were tighter. We could not configure the Witness from Topology unless we allowed the SQL Administrators to run Topology. You may be able to get your permissions granted as necessary, but just keep in mind that the backend permissions when a mirror and witness are involved are very delicate.
4.) Now to the biggest Gotcha I found. When deploying Lync Mirroring in Topology, we as Lync Admins are tasked with configuring a Mirror “Port” for each instance. The default is 5022 but this is configurable. Each subsequent Instance on the same SQL box, must have a unique Port, so the next one could be 5025 as an example. In the SQL world, this port is actually referred to as an “Endpoint”. It’s still using a port, it’s just that the terminology is used differently as a SQL DBA. If the endpoints are not defined by the SQL DBAs, we allow Lync to setup and configure the Port/Endpoint when we allow Lync to configure the Mirror. Allowing Lync to deploy the Mirror is our recommended approach as we are not DBAs(This is where the whole argument of who owns the mirror comes into play). During this setup stage the Mirror completes as expected and the port configured and we are happy. Now assuming we as the Lync Consultant had a temporary account created for us to deploy Lync, this is where the problem is. Inside of SQL, the Process ID (Owner of the endpoint) is the account used to create the Mirror. If I deployed the Mirror as Domain\TempLyncADmin that account now owns the Endpoint. When I leave as a consultant and they disable my account, that account is no longer valid, hence the Mirror will now not function. The Process owner must be changed from the Account who configured the mirror from Lync, to the ID of the Instance. My recommendation would be to have the SQL DBA configure the Endpoint prior to deployment and tell you want Endpoint/Port to use so they retain ownership.
Here is a snip-it from the SQL Architect I worked with explaining the situation:
“The first person who sets up sql server mirroring on an instance becomes the owner of the mirroring endpoints (this is the default behavior).
After setting up a mirror, the DBA will change the owner of the endpoint to the process account that runs the sql server services on the instance. We Do not want the endpoint owned by an individual’s login.
Once the ownership has been changed the service owners for all instances in this configuration (Mirror, principal, witness) are granted permissions to the endpoint.
(The permissions are granted at the time the endpoint is created. However since we change the owner, we re-grant the permissions.)”
Keep in mind moving forward that this process DOES NOT make the Lync Admin re-take ownership during Cumulative Updates to break and re-enable the mirror. This is just a one time ownership configuration.
If you are a consultant and you’ve moved on since a deployment, I would recommend you revisit this with your client. If you are an organization who did Lync yourself, I would recommend you review this and make sure your Endpoints are properly owned.