Optimising Your New Windows 2003 SP2 Domino Server

We recently bought a new server and replaced our trusty old main domino server. Some of my earlier posts covered this topic.

Our users were understandably expecting a bit of a performance boost - not that domino had ever under-performed, but hey, if you spend over $20K AU on a new server, you're entitled to expect to see some benefits.

Instead, the new server was slower than ever and we spent ages pondering the issue.

I've been rather busy of late, so this is the first server at my workplace which I haven't personally been involved in building. We got an outside party to do it. That's my excuse anyway - though admittedly, I doubt that I'd have done much better using my normal server-building routine.

Checking for the Cause
There are a few places you should check if ever you have a server which is under-performing.

• Event Viewer
  You should always have a browse through here to make sure that there are no messages from failed services or drivers. A driver that is failing or not working to it's full potential can place a serious drag on the system.
  

• Task Manager
  This is the fastest way to see what the server is doing. You can get to this with a single Ctrl+Alt+Del and a click on the task manager button. The first place to visit is the performance tab. If Either CPU or Page file usage is high, then you need to look at the processes and sort them accordingly. Eventually I'll do a post just on this.
  

• Performance Manager
  Like Event Viewer, this is found under Administrative tools. It will give you a lot of options for graphs. Ultimately, your aim is to determine whether anything flatlines at 100% though be careful as sometimes this is misleading. Eventually, I'll do a post on how to get benefit from this tool.
  

What it wasn't

• CPU and Memory
  It wasn't CPU, since CPU never seemed to go anywhere really. Domino is one of the most CPU friendly systems I know. It wasn't memory either. We noticed some issues with paging - it would sit at 100% for a time - regularly actually - almost like clockwork.
  

• Page Files
  We increased the size of the page file - for some reason Microsoft setup doesn't always allocate the correct size. The usual best for a page file is 1.5 times memory, so for a 4GB server, a 6GB page file is suggested. Of course, this isn't strictly true because the more memory you have, the less you have a need for a page file. One thing is certain, page files significantly over 1.5 times memory aren't used by the system. They're just a waste of space.
  
  
• Disk Issues
  Since the server was a newly loaded server, we could be certain that the problem wasn't fragmentation. Not so about the RAID controller though. We decided to split our seven drives into two RAID Arrays, a RAID 1 (Mirrored) set of 2 for the Operating System and a RAID 5 set of five for Domino and Data. I've heard that it's better for performance to put the Domino System itself on the mirrored drive but I've not had that confirmed.
  
  
• Notes Indexing
  When moving a Domino system to a new server, you need to expect to have poor performance until the full text indexes and the views are rebuilt. An overnight LOAD COMPACT task can help to take care of some of this. Mostly though, it's a matter of time. Our problem went on too long, so this wasn't the issue.
  

Dumb Microsoft Things
Microsoft does a lot of awful things to the Windows Server 2003 SP2 build. You need to undo these in order to improve performance.

• Indexing
  For a start, Microsoft turns indexing on for all drives by default. On a domino server which does nothing other than domino, this is sheer lunacy. To turn this off...
  
  
  1. Right Click on the Drive in My Computer and Select Properties.
  2. At the very bottom of the drive properties, deselect "Allow indexing service to..."
  3. Click on Ok.
  4. In the Confirm Dialog box, make sure you specify the drive and subfolders, not just the drive.
  5. Click Ok.
  6. I don't know if a reboot is necessary but it could help.
  
  Of course, you could just turn the indexing service off in service manager but I'd recommend disabling the flags as well in case a Microsoft Update accidental turns it back on.
  
  
  
  
  
  
• Data Execution Protection (DEP)
  In this case, this nasty little feature was the main cause of our troubles. I did a bit of searching and couldn't find any technotes to suggest that this feature should be turned off for Domino - though there's an IBM Note suggesting it needs to be turned off for QuickR.
  
  Interestingly, there are plenty of notes suggesting that it needs to be turned off for Exchange.
  
  To turn this baby off, you need to right-mouse click on My Computer, then choose Properties. Click on the Advanced tab and then on the settings button in the Performance frame.
  
  In the Performance options dialog box, click on the Data Execution Prevention tab and then choose the option to Turn on DEP for essential Windows programs and services only. Click on the button marked Ok. You will need to restart your server.
  
  

Resolving Network Performance Issues
In our case, a few PCs also had network performance issues. To sort these out, we turned on TCP/IP Port Compression. Here's how to do it.

For the Server

1. Open Domino Administrator
2. Click on the Configuration Tab
3. In the left hand pane, expand Server and click on Configurations.
4. In the middle panel Choose the server configuration you want to modify.
5. In the right hand pane, expand server then click on Setup Ports...
6. Click on the appropriate port (Usually TCPIP) but check to see that it's enabled to be sure.
7. Tick the checkbox marked Compress Network Data and then click Ok.
8. You need to restart the domino server for this setting to take effect.

Note that this setting will only produce benefit if other servers and clients have port compression enabled - otherwise it will send uncompressed data.





To turn port compression on for a Lotus Notes client;

1. Select from the Notes menu, File, Preferences, User Preferences.
2. In the User Preferences Dialog, choose ports.
3. On the ports screen choose the correct port (it will be the one that is enabled) and tick the box marked Compress Network Data.
4. Click Ok

You will need to restart your Notes Client for the settings to take effect.



One final note: Thanks to the guys at Convergence without whose assistance I probably never would have found these settings.