Friday, October 06, 2006

Unable to establish email address

I came across the following problem today:

Basically, after applying Windows 2003 SP1, only server local admins are able to establish SMTP addresses for objects (contacts, mailboxes, etc...). The error the user was getting was:
"An Exchange Server could not be found in the domain".

The jist is that non-local admins are not allowed to query for the status of the system attendant.


Tuesday, August 01, 2006

A closer look at DSAccess

Neil Hobson has posted an article to that covers in detail the dsaccess process. I learned that you can tell Exchange 2003 to not connect to a GC that is also a PDC. As Neil explains it, the PDC can become overwhelmed at times (password changes).

Key: HKLM\System\CurrentControlSet\Services\MSExchangeDSAccess\Profiles\Default
Value: MinUserDC


Wednesday, May 17, 2006

FSMO Role Placement

Good article on FSMO role lacement within AD.

Friday, March 31, 2006

What to do when a database won't mount

I came across the following site today while troubleshooting a database in the Recovery Storage Group that wouldn't mount.
What to do when an Exchange Store won't mount

The odd thing is that removing the database from the RSG and then re-adding it allowed me to mount the database without a problem.


Tuesday, March 28, 2006

Exchange 2003 Tunning - Back End Servers

I'm creating a build doc for all the Exchange 2003 mailbox servers. The servers will each hold 4000 mailboxes, and have 4 GB of memory. Here's what I'll be manually tunning:

1. Optimize Memory Usage
Heap Manager
- Minimizes VM fragmentation by increasing the amount of free space required before the heap manager frees up memory (default is 0)
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager
Value Name: HeapDeCommitFreeBlockThreshold
Radix: Decimal
Value Type: REG_DWORD
Value Data: 262144 (0x00040000 in hex)

Virtual Address Space
- 3GB allocates 3 GB of virtual address space to user mode. The number after userva is the amount of memory in megabytes (MB) that will be allocated to each process.
Edit the Boot.ini File.
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003, Enterprise" /fastdetect /3gb /userva=3030

2. Align I/O with Storage Track Boundaries (All SAN attached drives)
- Prevent a possible 20% performance hit due to track skipping
C:\>Diskpar –s drivenumber
Respond to both warnings by typing y
Please specify starting offset (in sectors): 128
Please specify partition length: [Pressing Enter will default to the max length]

3. Optimize NTBAckup
- optimize the data throughput.
HKEY_CURRENT_USER\Software\Microsoft\Ntbackup\BackupEngineIf BackupEngine is missing, run ntbackup once.
Logical Disk Buffer Size = 64
Max Buffer Size = 1024
Max Num Tape Buffers = 16

4. Improve Refresh time of mailbox configuration
- Mailbox limits are permissions will take effect faster (default is 2 hours)
Value name: Reread Logon Quotas Interval
Radix: Decimal
Value data: 1200 (20 Minutes)

Value name: Mailbox Cache Age Limit
Data Type: REG_DWORD
Radix: Decimal
Value data: 1200 (20 Minutes)

5. Move TEMP/TMP folders to RAID 1 partition
- Exchange uses TMP folders for mailbox moves

6. Increase ESE Buffer Size
- To optimize Virtual Memory useage by ESE, EXBPA recommends that servers with more that 2 GB of memory set the following:
Start the Active Directory Service Interfaces (ADSI) Edit utility.
Under Configuration Container, expand CN=Configuration, DC=example, DC=com.
Expand CN=Services, expand CN=Microsoft Exchange, expand CN=OrganizationName, expand CN=Administrative Groups, expand CN=First Administrative, expand CN=Servers, and then expand CN=servername.
Under CN=servername, right-click CN=InformationStore, and then click Properties.
In the Select which properties to view list, click Both.
In the Select a property to view list, click msExchESEParamCacheSizeMax
In the Edit Attribute box, type 311296 (1.2 GB)

7. Increase Transaction Log Buffers
- Increasing the size will provide better performance when multiple transactions are occuring (ideal for corporate environments). EXBPA recommends that if this value be changed to 9000.
Under Configuration Container, expand CN=Configuration, DC=example, DC=com.
Expand CN=Services, expand CN=Microsoft Exchange, expand CN=OrganizationName, expand CN=Administrative Groups, expand CN=First Administrative, expand CN=Servers, and then expand CN=servername.
Under CN=servername, right-click CN=InformationStore, right-click CN=, and then click Properties
In the Select a property to view list, click msExchESEParamLogBuffers
In the Edit Attribute box, type 9000

Thursday, March 23, 2006

Account Expires Attribute

I came across a posting today where someone wanted to set the account expires attribute to never using LDIFDE. It can be done using ldifde and using the ds tools.
LDIFDE or how I learned to love DS tools
C:\>ldifde -d "ou=test,dc=lab,dc=com" -s dcname -r "(&(cn=*))" -l accountexpires -f accExpires.txt

Here's what you'll get
----- Begin File: proxies.txt-----
dn: CN=Heras, Teo,ou=test,dc=lab,dc=com
changetype: add
accountExpires: 9223372036854775807
----- End File-----

Edit the file so it looks like this:
----- Begin File: proxies.txt -----
dn: CN=Heras, Teo,ou=test,dc=lab,dc=com
changetype: modify <---- change from add to modify
replace: accountExpires <---- This was added
accountExpires: 0 <----- this means never
- <---This is critical and the log file will tell you
----- End File -----

Finally, import the changes

c:\ldifde -i -f proxies.txt -s dcname -j c:-i means import, -j c:
DS Tools
dsquery user "ou=NoExpireDate,dc=lab,dc=com" | dsmod user -acctExpires Never

Wednesday, March 22, 2006

WMI Monitoring Script

I came into a situation where there are several Exchange servers without any monitoring. While software is procured, I created the following script to do some basic monitoring of Exchange services and disk space (to make sure circular logging doesn't kill the server). I have the script running as a scheduled task every 15 minutes. The script will create a log file every time it runs. If one of the thresholds is reached, an email is sent.

On Error Resume Next
Const ForAppending=8
Const ForReading=1
Const ForWritting=2
Dim strComputer
Dim objWMIService
Dim propValue
Dim objItem
Dim SWBemlocator
Dim UserName
Dim Password
Dim colItems

'Create Log file
Set objFSO = CreateObject("Scripting.FileSystemObject")
strPath = "C:\WMI Monitoring\"
strFileName = "server_status" & Hour(Now) & Minute(Now) & ".log"
strFullName = objFSO.BuildPath(strPath, strFileName)
Set objFile = objFSO.CreateTextFile(strFullName)
Set objFile = objFSO.OpenTextFile(strFullName, ForWritting)
'Build array of servers
arrServers = Array("exchange01", "exchange02")

'username and password
strUserName = "Administrator"
strPassword = "Password1"

For Each strComputer In arrServers
'WScript.Echo strComputer
ObjFile.writeline "===================================="
ObjFile.writeline "Computer: "& strComputer
ObjFile.writeline "===================================="
Set SWBemlocator = CreateObject("WbemScripting.SWbemLocator")
Set objWMIService = SWBemlocator.ConnectServer(strComputer,"root\CIMV2",strUserName,strPassword)
If Err.Number = "-2147023174" Then
strAlertItem = Err.Description
strAlertThreshold = "!!"
Call SendAlert(strComputer, strAlertItem, strAlertThreshold)
End If
If Err.Number <> 0 Then
objFile.WriteLine "Error Connecting: " & Err.Number & " " & Err.Description
End If

'Check Logical Disk
Set colItems = objWMIService.ExecQuery("Select * from Win32_LogicalDisk",,48)
objfile.WriteLine "Checking Free Disk Space"
For Each objItem In colItems
If InStr(objItem.Description, "Fixed Disk") Then
strAlertItem = objItem.DeviceID & ", " & objItem.Description
intFreeSpace = objItem.FreeSpace
intFreeSpace = intFreeSpace/1048576
strAlertThreshold = "Free SPace: " & CLng(intFreeSpace) & " MB"
'If there are less than 200 MB of Free Disk Space then send out an alert
If intFreeSpace < 200 Then
Call SendAlert(strComputer, strAlertItem, strAlertThreshold)
End If
objfile.WriteLine strAlertItem
objFile.WriteLine strAlertThreshold
objFile.writeline " "
End If
'Check Status of Services
Set colItems = objWMIService.ExecQuery("Select * from Win32_Service",,48)
objfile.WriteLine "Checking Exchange Services"
For Each objItem in colItems
If InStr(objItem.Displayname, "Exchange") Then
If InStr(objItem.Displayname, "Sync") Then
'WScript.Echo objItem.DisplayName
ElseIf InStr(objItem.Displayname, "Lotus") Then
'WScript.Echo objItem.Displayname
ElseIf InStr(objItem.Displayname, "Mailbox Manager") Then
'WScript.Echo objItem.Displayname
objfile.WriteLine "DisplayName: " & objItem.DisplayName
objfile.WriteLine "Name: " & objItem.Name
objfile.WriteLine "State: " & objItem.State
objfile.WriteLine "Status: " & objItem.Status
objfile.WriteLine " "
If objItem.State = "Stopped" Then
strAlertItem = objItem.Name & ":"
strAlertThreshold = objItem.State
Call SendAlert(strComputer, strAlertItem, strAlertThreshold)
End If
End If
End If
strAlertItem = " "
strAlertThreshold = " "
Set objWMIService = Nothing
Set objFSO = Nothing

'Send Alerts Via Email
Function SendAlert(strComputer, strAlertItem, strAlertThreshold)
'WScript.Echo "Sent Alert"
Set objEmail = CreateObject("CDO.Message")
objEmail.From = strComputer & ""
objEmail.To = ";"
objEmail.Subject = "Server Alert"
strText = strComputer & " is having the following problems: " & strAlertItem & strAlertThreshold
objFile.WriteLine "********************** ALERT SENT ********************************"
objEmail.TextBody = strText
objEmail.Configuration.Fields.Item ("") = 2
objEmail.Configuration.Fields.Item ("") = "nsmail01"
objEmail.Configuration.Fields.Item ("") = 25
End Function

Saturday, February 25, 2006

Microsoft Logos

I just found the following link on Microsofts site. If your certified in Microsoft technology, there's a new way to download logos.


Tuesday, February 21, 2006

How to recreate Exchange IIS virtual directories

Brian Posey wrote an article that is worth taking note of. He explains how to recreate the IIS virtual directories for OWA. This can be useful if there is corruption in the metabase, or if data deletion occurs to the files and folders needed.
Link to the article:,295582,sid43_gci1167561,00.html?track=NL-368&ad=541160

Overview of steps:
1. Backup IIS - This will ensure that further damage isn't done
2. Delete all the IIS virtual directories
a. Exadmin, Exchange, ExchWeb, Microsoft-Server-ActiveSync, OMA, and Public directories.
3. Delete the DS2MB metadata using Metabase Exploerer (IIS Resource Kit)
a. DS2MB stands for Directory Service to Metabase. It exists to bring over configuration information from AD to IIS. Remember that some OWA administration is actually done through ESM. Those changes come over with the help of DS2MB. I'm assuming that when DS2MB is deleted, the virtual directories are repopulated using the information in AD.
4. Restart the System Attendant and/or reboot the server to recreate the virtual directories.
5. Reset permissions on the ExchWeb virtual directory.
a. The article recommends enabling anonymous access and integrated Windows authentication on the ExchWeb directory. Anonymous access was already enabled when I tried this in my lab, and Integrated Windows Authentication was not needed.

KB Articles:
Overview of DS2MB
How to reset default virtual directories that are required to Provide Outlook Web Access, Exchange ActiveSync, and OMA

Monday, February 20, 2006

Delegating Admin Tasks

I'm a big fan of delegating only the admin rights that people need. It's easier to just give everyone full rights, but that's not very Elegant. Anyway, I read an article at today that corvers how to create a taskpad to delegate common administrative tasks.

Friday, February 17, 2006

HP Remote Management / ILO

Had an issue today where we needed to change the IP address of the ILO card. Normally, this can be done through one of two ways:
1. Through the ILO interface
a. https://iloipaddress
2. By rebooting the server and pressing F8

Neither of these methods was an option which allowed us to find a Utility from HP called "HP Lights-Out Online Configuration Utility." Its basically a command line tool that takes an XML file as input for ILO configuration settings. To get the utility to work, we had to install the following:
1. HP Proliant iLO Advanced and Enhanced System Management Controller Driver
2. HP Proliant Integrated Lights-Out Management Interface Driver
3. HP Lights-Out Online Configuration Utility
4. HP Insight Diagnostics Online Edition

Not sure which ones are needed, but the utility wouldn't work untill we installed all of the above.
Steps to change IP address:
1. C:\>hponcfg /w ilo_ip.xml - Exports configuration
2. Edit ilo_ip.xml to reflect new IP address
3. C:\>hponcfg /f ilo_ip.xml - Imports configuration

In the documentation, I also saw that this could be used to change the password.

Tuesday, February 14, 2006

Troubleshooting mail delivery and queues

Had an issue late this afternoon where the queue "messages awaiting directory lookup" had over 5K messages in it. I wanted to point to the following documents which detail how to troubleshoot each queue:
Troubleshooting Mail Flow and SMTP
Exchange Transport and Routing Guide
Modifying Logging Settings for MSExchangeTransport
Exchange Transport and Routing Guide

Basically, Queue buildup in "messages awaiting directory lookup" is related to AD connectivity. Here's a couple of ways to test AD connectivity:
telnet dcname 389 / 3268 (dc / gc)
lpd dcname 389 / 3268 (dc / gc)
dcdiag dcname

It turns out that one of the sites only has a single domain controller, which was probably overwhelmed. Lesson: Exchange needs at least two domain controllers local to it's site. Connectivity re-established by itself, but the queue continued to grow. We tried restarting the SMTP service, but it was stuck in a stopping state. There's a couple options available for this situation:
1. Force the smtpsvc to stop:
sc stop smtpsvc /force

2. Issue an iisreset /restart command which will bring down all the services related to inetinfo.exe (including SMTP).
IIS Library
It's important to note that if IISreset cannot bring down inetinfo.exe gracefully, then it will force it to stop. This can be avoided by providing the /noforce switch.


Friday, February 10, 2006


Consolidate Two Groups
I came across two queries that I wanted to pass along. The first allows you to consolidate groups. The query was posted by Mike Thommes who found it in a posting by Jerold Schulman. Here is my modified version

dsquery group -name GrpConsolidate **gets the DN of the target group
dsget group %groupdn% -members **get the members of the group
findstr /I /V /L /G: **regardless of case, print only lines that do NOT contain a match, and search strings literally, in file
dsmod group %groupdn% -addmbr **addms members to the group

Below is a combinations of these scripts to get everything to work.

c:\>dsquery group -name GrpConsolidate dsget group -members > c:\temp\target_group_members.txt

c:\>dsget group %sourcegroupdn% -members findstr /I /V /L /G:c:\temp\target_group_members.txt dsmod group <%targetgroupdn% -addmbr

The second query was from a request I made to the Exchange newsgroup on I needed to find a generic way to populate a group from a text file with SMTP addresses. Here's what Joe Richards (Joeware), and Michael B. Smith helped me develop:

Copy A Group
c:\>dsquery group -name [groupname]
-This will give you the dn of both groups

c:\>dsget group [sourcegroupdn] -members > c:\source_group_members.txt
- exports the DN of all the members to a text file

c:\>for /f %i IN (source_group_members.txt) do dsmod group [targetgroupdn] -addmbr %i
- parses through the text file and adds each DN to the variable %i, then the value of %i is passed to the dsmod query.


Create a group from a list of SMTP addresses

-- ReadSMTP.cmd --
for /f %%V in (smtp.txt) do dsquery * forestroot -q -filter "&(objectCategory=user)(proxyaddresses=smtp:%%V)" -attr distinguishedName >> UserDN.txt
-- ReadSMTP.cmd --

-- FillGroup.cmd --
for /F "delims=;" %%V in (userdn.txt) do dsmod group "[Group DN]" -addmbr %%V -q
-- FillGroup.cmd --

The same could be done using adfind and admod from I decided to use the ds* tools because they don't required the download and I wanted to keep it simple. Basically, I needed to hand this task off to another admin and I thought a script might complicate things.

How to use DSAdd to create multiple accounts.


C:\>for /f %i IN (userdn.txt) do dsadd user %i -pwd Password1

On a side note, if there is a space in the DN, you'll have to specify no delimeter (the default delimeter is a space)
C:\>for /f "delim=" %i IN (userdn.txt) do dsadd user %i -pwd Password1

Wednesday, February 08, 2006

Transaction Logs

If a backup is completing successfully, Exchange will flush all logs that have been committed to the database. So normally, all committed transaction logs will be flushed if:
1. All databases in the Storage Group are backed up
2. All databases in the Storage Group are mounted during the backup

The store determines what logs will be deleted by looking at the first log that has not yet been committed and deleting all log files previous to that. You can view the first uncommitted transaction log by running eseutil /mk on the checkpoint file.

After the backup completes, ESE Event ID 224 will be logged telling you what series of transaction logs will be deleted: If eseutil /mk E01.chk outputs E010000G then E0100005 - E010000F will be deleted. The purging process is sequential and will purge all log files in the series with one caveat - the purge process will stop if it goes to delete a log file that is missing. So in the above example, if log E010000A is missing, then only logs E0100005 - E0100009 will be deleted. In this scenario, after the next backup, Exchange will again try and purge all log files that have been committed. This time eseutil /mk E01.chk outputs E01000016 as the uncommitted log file and therefore E010000B - E0100015 will be purged.

If transaction logs are not purging, sooner or later you'll run out of disk space. If you have to create space in a hurry do not move the log files, compress them (in my lab I've seen 3 GB worth of log files compress to 1.5 GB). For recovery scenarios and for the purge process to complete successfully, do not move the transaction logs.

Transaction logging in Exchange server 2003:

Using ESEUTIL to determine which transaction logs have been committed:

How to remove Exchange server transaction logs:

Message Restrictions and Size Limits

Interesting read about how Exchange applies message restrictions / size limits.
Exchange Insider
It's important to note that for internal messages, the restriction setting on individual accounts trumps global settings.
For Internet email, global settings are applied: 322679


Tuesday, February 07, 2006

Free IPod...??

Hey [friend's name], Check out this site
Click here:

Wednesday, February 01, 2006

Designing Storage for Exchange 2003

Exchange Storage Design

The point of proper storage design is to ensure that each users gets enough IOPS. Consider the IOPS that the storage can provide:
* RAID 10 ( 2R 1W) RAID 5 (4R 1W)
* 15K rpm disks - 180 IO/second Before Controller
-Maximum Throughput = 180 X 80% (buffer) X .75 (RAID Factor) = 108 IO/second
-RAID 10 Factor = (R + W)/(R + 2W)
-RAID 5 Factor = (R + W)/(R + 4W)
* 10 Disks at RAID 10 - 1080 IO/second (Max IO/second that the disk will give you)
Using the example above, 1000 users would be able to receive 1.08 IOPS each.

JetStress can also be used to verify that 10 (15K RPM) disks at RAID 10 successfully sustain 1080 IOPS. Increasing the threadcount of JetStress will determine the most IOPS that those 10 disks can give you. As the thread count is increased, it's expected that the IOPS will not go above 1080.

Also, don't forget to consider IOs per database (since databases will be on seperate disks). If you pace 500 users on one database and 1500 on another, you will not achieve the required IOPS/user.

How many disks are needed?
To work this backwards we would need to following information:
* Mailboxes per server 1000
* Users I/O profile - 1.5 * IOPS = 1000 * 1.5 = 1500
*Read ratio(4 / (2+1) = .8
*Write Ratio (1 - .8) = .2
*RAID PENALTY for RAID 10 - 2 - Each write requires 2 disk I/O

(IOPS X Read Ratio) + [RAID Penalty](IOPS X Write Ratio)
Spindle Speed Behind Controller (180)

*Result = 1800 / 180 = 10

Exchange Team Blog - Disk Sizing
Exchange Team Blog - Disk IO
Petri - Exchange Sizing

Sunday, January 29, 2006

Exchange 2003 bandwidth requirements

I did some research over the weekend into the bandwidth requirements of
Exchange 2003 and Outlook 2003. Microsoft published an Operation paper titled "10 things to think about". Here they recommend using the following for a ballpark estimate:
Regular MAPI clients
# of concurrent users * 2.5 Kbps

Heavy MAPI clients
# of concurrent users * 3 Kbps

I noted some technology considerations below. I was also able to find
two white papers on traffic analysis; one by Citrix and the other by
Microsoft. Finally, I jotted down my initial thoughts.


Exchange 2003 Considerations
* LDAP searches directed at Global Catalog servers. By default, Exchange will refer Outlook clients to LDAP servers in the Exchange servers local site for directory access.
* Public Folder replication (unable to predict)
* OWA usage - Based on readings a single Outlook client, based on a
heavy profile, could require as much as 15 Kb/s per client (see white
paper below)

Outlook 2003 Considerations
* Every environment is different depending on the users profile (think of
IOPS when considering storage)
* Improved MAPI communication - Data is compressed and more data per
packet sent that in previous versions of Outlook. (only when working with
Exchange 2003).
* A single Outlook client, based on a heavy profile,
could require as much as 10 Kb/s per client (see white paper below)
* Cached mode has been designed to work over WAN links and tolerate
latency. However, it does require synchronization of an OST and OAB
(offline address book). This could result in high network utilization
if many users connect to Exchange at the same time and/or synchronize
their mail at the same time.

* OAB (Offline Address Book) - In a large organization, the OAB can be
several MB's in size (38 MB in my current environment). This can grow
significantly large when PKI is implemented. In addition there are multiple scenarios where Exchange forces all users to execute a full download:

Desktop search engines
* We've had some problems with the Google and MSN desktop
search engine. When these applications try and index a user's
mailbox, they'll execute as many connections as possible. Eventually one
desktop can put the load of 100 clients (just while indexing a single mailbox).

Citrix Traffic Analysis
Citrix Link

Microsoft Traffic Analysis
M$ Link

-- LINKS --
Top 10 things to Consider

One approach could be to work with the current Exchange adminstrators to
try and establish 3 basic user profiles. Then
hopefully make some basic bandwidth estimates and work with the network
team on creating a flexible network plan. As the migration
moves forward that plan could be adjusted based on trends.

Another approach could be to look at WAN accelerators


Tuesday, January 24, 2006


-- Mail Routing --
The routing of mail to Exchange recipients is done with the LegacyExchangeDN attribute. Messages sent are stamped with the senders LegacyExchangeDN.

-- Migrations --
With any version of Exchange, when the mailbox is moved, it's LegacyExchangeDN is overwritten to represent the new location in the directory. Because of routing explanation above, this breaks replyability. Normally, this is fixed by adding an X.500 proxy address that represents the LegacyExchangeDN. Exchange 2003 SP1 does this to all mailboxes moved between administrative groups.

-- Disaster Recovery --
A great article on the role that the attribute LegacyExchangeDN plays in a disaster recovery scenario.

Friday, January 13, 2006

Finding where a user was deleted

A blog I review posted 3 steps that show who deleted a user account.

  1. adfind -default -showdel -f (isdeleted=TRUE) -gc

  2. repadmin /showobjmeta dcname deletedobjectDN find /i "isdeleted"

  3. Eventcomb to find event id in security logs.

    • a. psloglist \\dcname security -i 630 -a date.


Thursday, January 05, 2006

Cached Mode, Security Groups, and Paged Pool Memory

The Exchange team's blog has an interesting article which goes into detail about security tokens, and how much paged pool memory they consume. If you have users in more than 80 security groups, it can impact performance. Read more by following the link below...