Microsoft Exchange 2000 with Dell SAN 4.0 - Part 1
By Richard Hou (Issue 3 2001)
This article provides a set of guidelines for deploying Microsoft Exchange 2000 Server on Dell PowerEdge servers in a Dell PowerVault Storage Area Network (SAN) environment. It also includes an overview of the Exchange 2000 architecture with a summary of key performance and scalability enhancements, a discussion of the Dell SAN 4.0 architecture, a configuration guide for implementing Exchange 2000 on Dell products, and performance metrics for tape backup and restore operations.
The demands for real-time messaging and information exchange increase dramatically with market globalisation. Applications that can handle efficient messaging collaboration—messaging exchange, group scheduling, electronic forms sharing, and multimedia conferencing—become even more important to the enterprise.
Microsoft® Exchange® 2000 is the first client/server messaging system that integrates all of these functions into a single system. However, enterprises that want to deploy Microsoft Exchange 2000 Server into an existing infrastructure must still address issues such as integration with Active Directory® , storage group design, data availability, server performance and scalability, and, most importantly, disaster recovery.
The Dell/Microsoft partnership has enabled the two companies to jointly provide integrated solutions, such as Microsoft Exchange 2000 into the Dell® PowerEdge® servers and PowerVault® Storage Area Network (SAN) products, which not only provide redundancy, but also provide leading price/performance for small, medium, and enterprise customers.
Microsoft Exchange 2000 architecture
Microsoft Exchange 2000, a major revision of Exchange 5.5, includes both enhanced features and newly developed functions. Architecturally, Exchange 2000 does not use a self-contained messaging system, thereby differing from Exchange 5.5.
Exchange 2000 separates many components, such as Active Directory Services, security, and Web access, then tightly integrates them with the underlying Windows® 2000 operating system and Microsoft Internet Information Services (IIS). For example, powered by the Web Store of Exchange 2000 and IIS, Exchange 2000 can host Web sites and workflow applications that use either Microsoft Outlook® or a data access browser, or those created by familiar application design tools, such as Visual Basic® or the newly developed Microsoft .Net. These improvements reduce the overhead on the Exchange servers, enable consolidation of the users onto fewer servers, and unify the network and messaging administration.
Figure 1 shows the Exchange 2000 architecture and the Exchange database implementation. The key features of Exchange 2000 that bring scalability, reliability, and availability include:
Figure 1. Microsoft Exchange 2000 architecture
- Active Directory integration. User manipulation, e-mail addresses, distribution groups, and other directory resources are stored in the directory database provided by Active Directory, a directory service running on Windows 2000 domain controllers. This simplifies user administration and reduces overhead on the Exchange 2000 servers by off-loading user authentication.
- Multiple message database support. Unlike Exchange 5.5, which used one single database for all users, Exchange 2000 allows the message store to be divided into multiple databases that can be managed either individually or in a logical group called a storage group. The message database can be located on one or more Exchange servers. Since every storage group can have its own transaction log, the repair or recovery of one database will not affect other databases in other storage groups.
Reliable SMTP support. Exchange 2000 uses the Simple Mail Transfer Protocol (SMTP) as the native communication method between Exchange servers for transferring and delivering e-mail. Compared with Remote Procedure Calls (RPCs) used by previous Exchange servers for message routing, SMTP provides major performance and reliability improvements.
Using SMTP, Exchange 2000 Server combines the Exchange Information Store and Microsoft IIS 5.0. A high-speed, shared-memory interface named Epoxy is the interface between the two services. Epoxy allows services hosted within the context of IIS (including POP3, IMAP4, NNTP, HTTP/DAV, and SMTP) to access information in the Exchange 2000 Store via their native protocols; therefore, they have direct access to the Information Store or the Web Storage System.
- Active/active clustering support. Exchange 2000 has enhanced cluster support that enables all systems in a cluster to actively process message requests. If one system fails, the entire workload can be redirected to the remaining node automatically while engineers recover the failed server. The end user experiences no interruptions and there is no need for a dedicated failover server.
Dell PowerVault SAN architecture
The Dell SAN uses Fibre Channel and RAID (redundant array of inexpensive disks) technologies to provide the speed, flexibility, and data storage capacity necessary to meet the performance requirements of today's complex server networks. This Fibre Channel storage subsystem can support large or small data storage networks. By using long-wave fiber-optic cable between cascaded switches, servers up to 10 kilometers (km) from the shared storage array can access data as if it were directly attached. Fibre Channel delivers up to 100 MB/sec of bandwidth for half-duplex configurations and up to 200 MB/sec for full-duplex configurations.
Dell SAN also provides a highly redundant environment. Using a redundant path from the front-end servers that share the storage equipment to the redundant switch between the storage subsystem and the servers will assure I/O access if one component fails. Battery backup guarantees the integrity of data written to the RAID controller caches. Dell provides two Fibre Channel Storage solutions:
- Dell PowerVault 650F/630F. PowerVault 650F/630F is the first storage area network solution from Dell. PowerVault 650F is a 6.5U disk processor enclosure (DPE) with dual-active RAID controllers that support up to ten 3.5U PowerVault 630Fs. It supports 9 GB, 18 GB, 36 GB, and 73 GB drives. A fully populated PowerVault 650F/PowerVault 630F can support up to 8 TB of data.
- Dell PowerVault 660F/224F. The PowerVault 224F Fibre Channel storage array provides scalable, rack-dense RAID storage with up to 14 one-inch Fibre Channel drives in a 3U enclosure (one rack unit = 1.75"). The PowerVault 660F is a PowerVault 224 with one or two RAID controller modules. These RAID modules in the PowerVault 660F can manage its 14 disks and disks in up to seven PowerVault 224Fs for a total storage subsystem of 4 TB. With the new release of 73 GB drive support, a single storage subsystem can handle up to 8 TB of data.
Implementation and performance testing
The test objective was to provide a set of guidelines for deploying Microsoft Exchange 2000 Server on Dell PowerEdge servers in a Dell PowerVault SAN environment. Using the hardware and software described below, simulated workload was used to generate as close to a real environment as possible.
In addition, the testing involved various configurations for implementing Exchange 2000 on Dell SAN and server products. The results provide considerations and suggestions for achieving better performance through performance monitoring and storage I/O planning. An analysis of the tape backup/ restore data offers guidance and relative performance metrics for customer planning.
Hardware and software
The test configuration consisted of two Dell PowerEdge servers, each with redundant power supplies in case of hardware failure on one power supply:
- One Dell PowerEdge 6450 as the Exchange 2000 server. The PowerEdge 6450 Exchange server configuration included two Intel® Pentium® III 550 MHz/2 MB L2 cache CPUs, 4 GB memory, and VERITAS® Backup Exec® 8.0.
- One Dell PowerEdge 4350 as the domain controller to serve Active Directory Services. The directory services placed on a PowerEdge 4350 (two Pentium II 450 MHz processors and 2 GB of memory) helped reduce overhead on the Exchange server. The PowerEdge 4350 server handled user authentication and related directory services.
Ten Dell OptiPlexTM desktop computers provided client simulation of multiple virtual users for different scenarios. One Dell PowerVault 660F and one Dell PowerVault 224F served as the Fibre Channel storage subsystem for the Exchange database. Both had 14 one-inch 10,000 rpm Fibre Channel disks; the PowerVault 660F used two RAID controllers, each with 512 MB onboard cache.
For storage subsystem connectivity, the Exchange server had two QLogic® QL2200 Host Bus Adapters (HBAs). Two Dell PowerVault 51Fs created a switched fabric. Also, a Dell PowerVault 130T served as the tape backup library, connected by a Dell PowerVault 35F Fibre Channel-to-SCSI bridge.
Figure 2 shows the complete configuration tested, with a full redundant path for high availability. Figure 3 shows the software and drivers installed on the Exchange 2000 Server.
Figure 2. Exchange 2000 Server configuration
Figure 3. Testing software and driver versions
The Microsoft Messaging Benchmark 2 (MMB2) specification provided the standards for the user workload and Microsoft LoadSim generated the user database and messaging requests.
We created databases of 500, 1,000 and 2,000 users on a RAID-10 array with 8 KB, 32 KB, and 64 KB stripe size. Then we collected backup and restore times based on different user environments.
Figure 4 shows detailed information about the MMB2 configuration. For additional information, visit http://www.microsoft.com/Exchange/techinfo/planning/2000/E2KDell640082k.asp.
Figure 4. MMB2 configuration for the PowerEdge 6400
Exchange database implementation
The database implementation of Exchange 2000 Server differs significantly from its predecessors. To understand the Exchange database, a few concepts need clarification. First, the Information Store (IS) is the central repository for all messages on a Microsoft Exchange Server. Information is typically maintained in two databases: the Public Information Store and the Private Information Store. Previous versions of Exchange have only one storage group, containing a single Private Information Store and a single Public Information Store.
In Exchange 2000 Server, the Information Store provides new methods for accessing Exchange data. Since Exchange 2000 supports any Windows or Internet client and various application programming interfaces (APIs), it exposes data through MAPI, HTTP, and Win32® . This enables the underlying data to be accessed through a drive letter, Web browser, or a universal naming convention (UNC) path. The Exchange 2000 Information Store, therefore, becomes a Web Store.
Exchange 2000 Server supports a maximum of four storage groups with up to five databases per storage group. Each storage group uses a common log file and runs as a Joint Engine Technology/Extensible Storage Engine (JET/ESE) service in the process store.exe.
Figure 5 shows the database architecture for Exchange 2000. This implementation reduces the size of individual databases, thereby allowing faster backup and restore and increased flexibility in managing backups. Note that in Figure 5 the transaction log files are labeled "Uncommitted entries" because the current database combines the database file and the uncommitted log file. The transaction log, a copy of the memory content, is important because it reflects what will be changed in the database. It should be placed on a reliable and high-performance RAID array.
Figure 5. Exchange 2000 database architecture
The Dell PowerVault 660F provides onboard cache of up to 512 MB per RAID controller. The cache is write-mirrored protected when the partnership is enabled and configured as Normal Cache Mode. When the write-back cache is enabled, it is best to use RAID-10 for the transaction log.
I/Os for RAID levels
It is important to understand the I/O nature of different RAID levels and the Exchange Server Information Store configuration before designing an Exchange database. Although RAID levels 1, 5, and 10 provide redundancy, the I/O performance differs among the levels. All levels use the same amount of I/O per read, but for every logical write, RAID-5 uses four physical I/Os for the operation compared to RAID-1 and RAID-10, which use two I/Os. For example, RAID-10 provides better performance than RAID-5 for a write-intensive database. Figure 6 shows the various I/Os for different RAID levels.
Figure 6. I/Os for different RAID levels
Figure 6 can be useful for calculating the number of drives in the RAID array. For example, the PowerVault 660F/224F can use the Seagate® IV 10,000 rpm or 15,000 rpm drives. Generally, the 10,000 rpm drive produces 100 random I/Os per second. By monitoring the current messaging environment or setting up a pilot test, data can be collected from the Microsoft Performance Monitor object PhysicalDisk under read/sec and write/sec. If the configuration uses RAID-10, the following formula will calculate the number of RAID drives:
Number of drives = 2 x (Read/sec) + (2 x Write/sec) / (I/Os per second per drive)
Figure 7 shows I/O characteristics of the Exchange database file and the VERITAS Backup Exec (used as the tape backup application in the tests). Although Exchange 2000 Server uses random I/O and a 4 KB transfer size, VERITAS Backup Exec uses 64 KB sequential read/write to back up and restore data. Therefore, the performance of 64 KB sequential read/write should be considered equally with 4 KB random read/write when choosing the RAID level for the mail server.
Figure 7. I/O profile for Exchange 2000 database file and backup software
Tuning for database performance
Decreasing response time yet supporting more users is the goal in tuning the Exchange database performance. The I/O pattern for the Exchange database (random reads and writes with a 60/40 split) differs from the transaction log I/O pattern. Although read cache is not significant for database access performance based on this pattern, use of the write cache will lower response time.
The RAID controller in the PowerVault 660F distributes the read/write cache dynamically. To support more users, the disk array should include more spindles to handle multiple concurrent user requests. Currently, Dell SAN 4.0 with PowerVault 660/224F supports up to 16 drives in each virtual disk.
Other factors involved in sizing the Exchange database include determining the number of users to be supported and identifying how large each user's mailbox can grow. This number determines the total space required for the database file. Once this number is defined, Microsoft recommends doubling the capacity of the database file.
Two benefits from this process include:
- Disk maintenance like defragmentation can be performed quickly without copying the database file to a separate drive.
- The extra space helps to reduce the time required to restore the database from tape.
Once the total capacity is identified, divide it by the capacity of a single drive to determine the number of drives required. For RAID-10, double this number; for RAID-5, add 1.
Dell SAN 4.0 with PowerVault 660F/224F supports the Online Capacity Expansion (OLCE) and Online Virtual Disk Expansion (OLVDE). The array can handle added capacity and extra spindles without stopping the environment.
Hardware stripe and NTFS block size
The hardware stripe size and NTFS block size are important factors to consider. By default, PowerVault 660F uses a 64 KB stripe size for the RAID, which provides optimum performance for most applications.
These tests used 8 KB, 32 KB, and 64 KB stripe sizes for the Exchange database. The results showed that the 32 KB stripe size performed better than 64 KB in the Exchange environment. Figures 8 and 9 show the LoadSim score and the I/O latency for different configurations.
Figure 8. LoadSim score for 2,000 users
Figure 9. I/O performance for 2,000 users
The default allocation unit size provided by NTFS depends on the volume size. Matching the allocation unit size with the amount of data typically transferred by the application to and from the disk will lower disk subsystem overhead and improve overall performance.
Figure 10 shows the Performance Monitor result from the lab test. Data collected from the PhysicalDisk object on both log file and database drives included Average Disk Bytes/Transfer, Average Disk Bytes/Read, and Average Disk Bytes/Write. The average transfer size is about 4 KB, which will be the block size for NTFS file system.
Figure 10. I/O profile Exchange 2000 database and log file
Figure 11 summarizes the recommended configurations for RAID level, stripe size, and NTFS block size for the various files in the Exchange 2000 database. The size of the tested database determined the number of spindles (Exchange users should determine this for their own specific environments based on the algorithm above).
Figure 11. Exchange 2000 database: log file layout parameter recommendations
For these tests, we placed the .edb file and .stm file on the same drive, since the MMB2 workload is a Microsoft Outlook-based user workload that will simulate Messaging API (MAPI) users without touching the .stm file (streaming media store). If part of the Exchange workload involves streaming media, it is best to place the .edb file and .stm file on two separate physical drives since the I/O of these two files differs.
Based on observations from testing, the final configuration for the database is one storage group with two databases: The first database contains 100 user mailboxes and the second has 1,900 users. The Backup/Restore section discusses the reason for separating the database into two files.
Storage subsystem management
Dell SAN 4.0 and the PowerVault 660F use the Dell OpenManageTM Array Manager for the storage subsystem management. To configure the RAID controller card, first start the Array Manager, expand the PowerVault 660 system on the left panel of the console, expand the Physical Array, right-click on the controller and click Controller Options. These applied changes improve performance:
- Conservative cache mode disabled. This is enabled by default so the controller cache is not being used for I/O buffering.
- SES communication drive as a hot spare. A SCSI Enclosure Service (SES) communication drive is located in either slot 1 or slot 2 of the PowerVault 660F enclosure. Each enclosure has only one SES communication drive. Making the SES communication drive part of the disk group may cause performance degradation of the disk array (where the impact would lessen with more disks in the array). Therefore, it is better to leave this drive as a hot spare if not needed in the array.
Backup and restore strategies
Disasters such as hardware and software malfunctions can cause loss of data. Enterprises need a robust backup/restore strategy to protect key data and to enable them to get their Exchange environment back to normal quickly and efficiently.
Recovering Microsoft Exchange 2000 Server data efficiently requires a good understanding of Exchange 2000 database technology. The Exchange 2000 model supports a single server running multiple private and public databases. Multiple database files (MDBs) enable separate databases to be backed up and restored independently without shutting down the rest of the messaging system.
Backing up files for Exchange 2000 and Windows 2000
The following Exchange 2000 and Windows 2000 files should be backed up:
- Information Store including Private Information Store, Public Information Store, and transaction logs
- Windows 2000 operating system Registry for service-related and configuration information
- Other files and directories used by Exchange 2000, such as the MTADATA directory that contains messages and transits through the MTA, and customized scripts for the scheduled jobs
- Application and backup software, such as Exchange 2000 itself and VERITAS Backup Exec
In addition to general backup methods of online and offline, Exchange 2000 Server supports different types of backups: full, copy, incremental, and differential. The importance of the data determines the type of backup to perform. Each backup type has its advantages and disadvantages in terms of data storage, performance, and time requirements, shown in Figure 12 .
Figure 12. Exchange 2000 database backup methods
Tape backup requires Exchange-aware backup software. Our tests used VERITAS Backup Exec 8.0 (part of the Dell SAN backup solution) for Windows NT/2000.
Full backup is the preferred method since it makes full backup copies for both Exchange database and Exchange log files. It also deletes those transaction log files that contain transactions committed to the server database. Restoring from a full backup is always easier because it usually involves only one backup tape.
Several important points to consider before designing the backup/restore plan include:
- Always try to include the Windows 2000 Registry. A backup of the Windows Registry makes it easier to replace the Registry during a restore.
- Although databases can be backed up individually, backing up an entire storage group at one time allows the transaction log file with each storage group to be backed up only once.
- Only one backup or restore process at a time can occur in a single storage group. Multiple processes can be scheduled for databases within different storage groups.
The testing focused on the length of time to perform online backup/restore while MMB2 workloads of 500, 1000, and 2000 Exchange users were running. The test used a Dell PowerVault 130T tape backup library connected to the PowerVault 51F Fibre Channel switch through a PowerVault 35F SCSI-Fibre Channel bridge.
Generally, the restore time defines the Service Level Agreement (SLA) time, so these tests attempt to show the restore time with different numbers of users. The main database (with 1,900 mailboxes) was backed up to tape, and the separate database file (with 100 mailboxes) was separately backed up to a local disk using Windows 2000/Exchange 2000. This smaller, easily restored set of mailboxes could be used for the enterprise's most critical set of users.
Figure 13 shows the various backup times for different numbers of users. From the data, a storage group with 2,000 users can be backed up within 2.5 hours. To avoid long backup times, extra users should be separated into separate storage groups on a separate disk array.
Figure 13. Backup times to tape for 500, 1,000, and 2,000 users, and to disk for 100 users
Restoring databases with Exchange 2000 running
The type of disaster determines the recovery process: restoration of a single mailbox, database, storage group, or the entire Exchange 2000 Server with a previous configuration. This test focused on the length of time to restore different databases with various numbers of users while the Exchange 2000 Server is still functional.
The best approach to restore databases is to stop the Exchange application and dismount the databases to be restored; however, the Web Storage System must continue running. It is extremely important to make a copy of the current database—even if it might appear corrupted—and not delete this copy until the database is restored and verified. If the restore fails, the copy might be used for repair.
When the restore begins, Exchange Server will dynamically create a storage group to save the database and transaction log files from the backup. During restoration, the transaction logs are applied to the database in the restore storage group. After recovery, Web Storage System mounts the recovered database into its original storage group.
Only one backup or restoration process on a particular storage group can run at a time. To run the backup and restore simultaneously requires configuration of multiple storage groups that support large numbers of users. By splitting the mailboxes into multiple databases, a server can support more users without each backup/restore process requiring a long time.
Two methods can be used to restore multiple databases in different storage groups:
- If multiple databases are selected when scheduling the restore, the restoration will restore these databases simultaneously by creating a restore.env file for each restore process.
- For one-by-one restoration, each database must be restored and mounted before the next database restoration can be started; otherwise, the restore.env file for the second job will erase the previous restore.env information without the first one being done. See Microsoft Exchange 2000 online documentation and the VERITAS Backup Exec online help for detailed information.
Figure 14 shows the restore time for the 500, 1000, and 2000 user environments as well as for the 100 user restore direct from disk.
Figure 14. Restore times from tape for 500, 1,000, and 2,000 users and from disk for 100 users
Dell SAN 4.0 and Exchange 2000: A good partnership
The tests show that the PowerEdge 6450 server and PowerVault 660F/224F running in a Dell SAN 4.0 configuration are well suited to run Exchange 2000 in an enterprise environment. Exchange 2000 Server takes full advantage of the high availability and scalability delivered by this configuration. With up to 4 TB storage space (soon to 8 TB) and configuration flexibility, the PowerVault 660F/224F makes it easy to expand the size of the Exchange database.
With the right design, configuration, and continuous monitoring, Dell servers and storage products combined with Microsoft Exchange 2000 create a robust messaging environment.
Richard Hou (firstname.lastname@example.org) is a systems engineer/consultant on Dell's Solutions Enablement Labs and Showcase Team, part of the Dell Enterprise Services and Support Group, specializing in SAN and Microsoft solutions. Richard has a Masters degree in Electrical and Computer Engineering from the University of Texas at Austin and a Bachelor's degree in Mechanical Engineering from Zhejiang University, Hangzhou, China.
For more information
Microsoft Exchange Web site: http://www.microsoft.com/exchange
Dell online: http://www.dell.com
Stanek, William R. Microsoft Exchange 2000 Server Administrator's Pocket Consultant . Redmond, WA: Microsoft Press, August 2000
Goncalver, Marcus. Exchange 2000 Server Black Book: A Guide to Implementing and Supporting Microsoft's Newest Version of Exchange . Scottsdale, AZ: The Coriolis Group, LLC, July 2000
Microsoft Exchange 2000 Server Resource Kit. Redmond, WA: Microsoft Press, September 2000