Blackberry Server 5 and Exchange MAPI CDO, and Windows 2008 Domain controller

It has beed nothing but fun after I upgraded BES 4 to 5.
Most painful problems are :
besadmin keeps loosing connections to exchange server.
After BES 5.0 SP1 gets installed, I could not login to BES admin web site.

I opened a ticket with RIM and spent almost a month with them hoping they could figure them out and finally I figured out and fix them by myself.
I like to share how I fixed it so you don't have to spend a month with useless RIM support.

First check your MAPI CDO's version if it is either 6.5.8147 or 6.5.8153, you need the newest MAPI CDO. Those MAPI CDO has unexpected crash problems. Check this website for more information: http://blogs.msdn.com/b/stephen_griffin/archive/2010/04/29/updated-mapi-download-fixed-crashes.aspx

and if you upgraded domain controllers to Windows 2008 servers, GC will refust many connections from the BES and resutling slow email delivery and many pending messages on BES.

This is part of the error on BES :  {0x4EC} CDO initializing failure in CDO helper
The information store could not be opened. [MAPI 1.0 - [MAPI_E_LOGON_FAILED(80040111)]]


To fix this issue, take a look at this M$ KB2019948

http://support.microsoft.com/kb/2019948

Windows Server 2008 GC’s have a limitation of 50 concurrent NSPI connections per user.
Applications such as Outlook 2007, Blackberry Enterprise Server, or any application that will create multiple NSPI connections to a DC/GC can experience these symptoms.
 


-To fix login failure issue after BES 5 SP1 gets installed


See RIM KB20406

Here is the SQL script to fix the particular issue, run it against BES 5 BESMGMT database

update dbo.BASTraits
set value = null
where traitid=0 and pluginid=111

How to get copy of NDR sent to your email address

On EMS run this command

Set-OrganizationConfig -MicrosoftExchangeRecipientReplyRecipient


Set-TransportConfig -GenerateCopyOfDsnFor

For example

Set-OrganizationConfig -MicrosoftExchangeRecipientReplyRecipient xyz@abc.om

Set-TransportConfig -GenerateCopyOfDsnFor 5.1.1,5.7.1,5.7.2,5.7.3

For more information about DSN messages: http://www.computerperformance.co.uk/exchange2007/exchange2007_ndr.htm

Installing Exchange 2010 - Part 6

Activating the Mailbox database copy.

From the Database Management, select Add Mailbox Database copy.



Select a target mailbox server




Adding..






Added



Verify the completion



On EXB2, mailbox copy has started.


Do the same for the EXB2/DB2


Isn't this easy?

Installing Exchange 2010 - Part 5

Let's setup a DAG

I will name the DAG as "DAG1" and its IP will be 192.168.1.10, and use EXF1 for the FSW.

Creating DAG with EMC is a bit problomatic, use powershell instead.

If you use EMC, DAG will grab a DHCP IP.

New-DatabaseAvailabilityGroup -name DAG1 -WitnessServer EXF1 -WitnessDirectory c:\dag1fsw -DatabaseAvailabilityGroupIpAddresses 192.168.1.10

Now DAG1 is created but there are no members

Let's add members

I will add EXB1 and EXb2.

They are added, from the database management tab, you can see the owner of the DB. currently EXB1 is the owner of the DB1 and EXB2 is the owner of DB2.

Next time we will activate Mailbox Databse Copy.

Installing Exchange 2010 - Part 4

Eventhough Microsoft does not recommand using WLB on Exchange 2010 servers, I will show you how to do it. Microsoft actually recommand using H/W load balancer such as F5.

I will assume you already have experiences on creating NLB on exchange 2003/2007 servers.

Installing WLB on CAS&HT servers.

1. Install "Network Load Balancing" feature on each CAS+HT servers.

2. Setup the NLB (in my case my nlb name is "webmail.demo.local"), exf1 and exf2 are the members, also add it to DNS.

3. set a new client access array using new-ClientAccessArray command, it will be used among outlook clients (MAPI) to connect to the exchagne 2010 servers.

4. Now associate the array with a database using set-mailboxdatabase command

Now when db1 user try to connect to exchange database using mapi protocol, they will be directed to use webmail.demo.local to connect.

Some admins may perfer using a different name for this, for example "outlook-connect" or etc.

To do that,

Add A record for the "outlook-connect", but use the same nlb cluster IP (webmail's IP in my case)

Create a new ClientAccessArray, and associate a DB with it.

Let's verify the setting.

Installing Exchange 2010 - Part 3

The two servers I will use to install CAS & HT servers are called EXF1 and EXF2.

I will skip the prereq processes (eg: .net framework and etc)

1. For CAS and HT servers run following commands from the scripts folder/directory.

sc config NetTcpPortSharing start= auto

ServerManagerCmd -ip Exchange-Typical.xml -Restart

Click this link for more details on the command

2. Install exchange server, choose custom install and select CAS and HT roles.

3. Populate the internet facing server name (if your CAS&HT server will face the internet)

4. Install

5. Installed, internal URL will be changed once the load balance feature gets installed.

6. I will skip the installation process for the EXF2 (my second CAS&HT server)

Installing Exchange 2010 - Part 2

Name of the server I will install the first mailbox server is exb1

1. Start the setup, and choose language option

2. And install exchange server

\

Select mailbox role

3. First look of the EMC

Installing Exchange 2010 - Part 1

I am going to install exchange server 2010 on a server 2008 SP2, for more information on Exchange 2010 Prerequisites, see http://technet.microsoft.com/en-us/library/bb691354.aspx

1. Install .net Framwork 3.5 SP1, you can download the full package version from here

2. Install Windows Remote Management (WinRM) 2.0 and Windows PowerShell V2, download it from here

3. On servers that will host the Hub Transport or Mailbox server role, install the Microsoft Filter Pack. Download it from here

4. Run "ServerManagerCmd -ip Exchange-MBX.xml -Restart" from the scripts folder on a exch 2010 installation CD.












5. Prepare schema by running "setup /PrepareSchema"










6. Prepare AD by running "setup /preparead" (setup /PrepareAD will automatically perform the PrepareLegacyExchangePermissions )

7. Prepare domain
Run setup /PrepareDomain or setup /pd to prepare the local domain
Run setup /PrepareAllDomains or setup /pad to prepare all domains in your organization.

Exchange 2010

Exchange 2010 is out and there are a few information about how to create/setup high availablity features.

I am going to post how to setup exchange 2010 with DAG and setup load balanced CAS and HT servers.

Outlook calendar pemission getting reset issue

There is a case when a user wants to block his/her calendar (free+busy) information not to be announced to users in your organization.
They can block it by selecting calendar's permission to "None" for the default setting.
It gets set back to "free+busy" after about 24 to 48 hours later. it is done by BES (4.1.5 or later).
Disable the free+busy look up capability to prevent the problem.

Here is an instruction:

http://www.blackberry.com/btsc/search.do?cmd=displayKC&docType=kc&externalId=KB15162&sliceId=SAL_Public&dialogID=91835014&stateId=0%200%2091831361

* if you do it, users with handheld software version 4.5 or later can not perform free+busy lookup

Update Rollup 4 for Exchange Server 2007 SP1 is available

Update Rollup 4 for Exchange Server 2007 SP1
http://support.microsoft.com/?kbid=952580

There are a few possible installation issues that we would like you to be aware of:
1. Exchange 2007 managed services might time out during certificate revocation checks
2. When installing a Rollup, we recommend you use the same account that you used to install Exchange Server. If you are using a different account, that account needs to have Local Administrator rights as well as rights to read Active Directory on Exchange object as well as server level (as the update needs to determine which roles are installed on the server). Not having required permissions can lead to OWA not being updated correctly and displaying a blank page after update has completed.
3. If you have modified the logon.aspx file, it will not be patched by the Update Rollup installer. As a result Outlook Web Access may not be updated correctly and it may display a blank page after the update has finished. In order to avoid this problem, rename the logon.aspx file before applying the update rollup. After you apply an update rollup package, you must re-create Outlook Web Access customization in logon.aspx.

A Note:
If you have Cisco Unity and exchange server 2007, you might experience IP phone's LED (Voicemail indicator lamp) won't syncronize when a user receives or deletes a message. This update, kb952580, will fix the issue.
This update rollup 4 for exchange 2007 sp1 includes 951251 (http://support.microsoft.com/kb/951251/): A MAPI application does not work correctly if Exchange 2007 is installed on a Windows Server 2008 server.
More information on what was really broken, take a lok at http://blogs.msdn.com/stephen_griffin/archive/2008/07/03/mapi-download-and-the-enumprotocols-bug.aspx

Also If you are using Blackberry enterprise server 4.1.3 or later with exchange server 2007, you might want to update CDO&MAPI on BES, this is optional, here is an instruction
http://www.blackberry.com/btsc/search.do?cmd=displayKC&docType=kc&externalId=KB14502&sliceId=SAL_Public&dialogID=192826523&stateId=1
On this instruction remove ESM 2003, and follow the instruction in full.

Migrating exchange 2003 clustered server environment to exchange 2007 sp1 using windows 2008 servers

Before I post detailed steps, here is a migration plan.

The 2x E2K3 OWA (NLB) and 2x E2K3 clustered backend servers will be replaced/migrated to

• 2x E2K7 SP1 CAS&HT (NLB)
• 1x E2k7 sp1 public folder server (LCR)
• 2x E2k7 sp1 (CCR) mailbox servers (CCR: 1 active and 1 passive)
  
  0. Prep 2x windows 2008 server STD for CAS and HT, install IIS, NBL and etc.
  1. Export IIS cert from the exchange 2003 OWA server, and import that to the windows 2008 STD servers.
  2. Install exchange 2007 sp1 on windows 2008 server.
  3. Create/modify E2k7 connectors, create routing connector to connect EVS1 and E2k7 CAS&HT server.
  4. Remove e2k3 OWA servers.
  5. Setup e2k7 sp1 STD public folder server (single mailbox server withot mailbox DB)
  6. set OAB on e2k7 public folder server for old outlook users. replicate Public Folders.
  7. Prep CAS&HT for witness server.
  8. install e2k7 ENT CCR mailbox server.
  9. migrate mailboxes
  10. remove e2k3 server clusters.
  11.Clean up AD using ADSI edit tool.

It has been a while

It has been a while since I posted the last post here, I have been busy figuring out how to migrate exchagne 2003 + windows 2003 R2 sp2 to Exchange 2007 sp1 on windows 2008.
It is going to be a very interesting post.
Check out this blog soon!

How to enable Forms Based Authentication on Exchange 2003 OWA

I will assume you know how to use IIS and request and install SSL.
1. On one of the front end server, request and install SSL.
2. Open the IIS manager, find a "exchange" virtual directory and set Require SSL and Require 128 bit encription.
3. On Exhange Systems Manager, open the front end server.
4. Go to the protocol--> http, right click the http and open the properties for the Exchange Virtual Server.
5. Click settings tab, and check the "Enable Forms Based Authentication"
6. On the front end server, reset the iis or restart the server.
7. Try connect the owa using https://
8. On the front end server, start the IIS manager and export the cert.
9. on the second front end server, import the cert.
10. repeat #2 to #6 on the second front end server.

Migrating exchange 2003 clusters to Exchange 2007 clusters

Since exchange 2007 become more popular I decided to start yet another series. In this series, I will show you how to:
1. How to use VMware virtual server to install 64bit OS, exchange server 2003 cluster.
2. Prep exchange 2003 for testing.
3. Install first exchange 2007 (CAS and HTR)
4. How to use 2007 OWA to access mailboxes in exchange 2003.
5. Install exchange 2007 NLB.
6. Install exchange 2007 back end clusters.
7. Migrating things
8. Removing exchange 2003.

Step by step: Installing Antigen on Exchange 2003 cluster

1. On a active node, yes active node, install antigen 9.0


2. In the middle of setup it will ask for a shared disk's drive letter, don't put drive letter (exmaple X:), instead put name of the sharted disk, in our lab that is "Disk x:".

If you put actual disk letter (X:) you will receive

Failed to get EVS Info.GetMSCSEVSInfo: An Active node of an Active/Passive MSCS cluster is found, but not match for the shared drive is found.




3. Follow the instructions and finish.

4. Install Antigen on every active node.

5. Install Antigen on every passive node.

6. Done.

Step by step: How to install exchange 2003 cluster part XIIII

Simulating the quorum disk failure

1. Turn off both EXBE01 and EXBE02.
2. Edit each machine and remove the Quorum disk (Q:).
3. Start EXBE01 and EXBE02.
4. Cluster service won't start.
5. Turn off both EXBE01 and EXBE02.
6. Add a new SCSI shared disk.
7. Turn on EXBE01.
8. Format the new disk and lable it is as S:.
9. ON EXBE01, go to the "service" change "cluster" service to "manual".
10. From the command line execute "net start clussvc /fixquorum" .
11. Start the cluster admin GUI and add a new resource (physical disk) S: and assign it as a new quorum disk, and make groups online.
12. Change cluster service to "Automatic" start.
13. Reboot EXBE01, and check to see if the cluster is running.
14. Start EXBE02.

Step by step: How to install exchange 2003 cluster part XIII

How to add more than 2 nodes.

1. You can't simulate this in this lab because MS vitrual server does not allow more than 2 channels on SCSI.
2. To do it on real world, keep both current nodes on.
3. On the third node, install OS, apply server service packs and patces.
4. Add it as a node, just follow steps as if you are adding the second node.
5. Install exchange server, add groups, this time EVS2.
6. Apply exchange SP, hot fix, security paches and so on.
7. EVS2 is set.
8. You now have 2 active and 1 passive node, make sure all nodes are possible ower of all the resources.
9. Keep doing this to add more active nodes, you can add up to 8 nodes (At least one node needs to be a passive)

If you don't create EVSx, that will be just a passive node.

* Make sure all nodes are identical, for example BIOS, iSCSI initiator, SCSI firmware, driver, exchange hotfix, virus scanner and etc.etc.................I mean everything!

To add third node in this lab. - I will be really brief
1. Don't try this unless you are really curiouse about how to do it.
2. Turn off the passive node. (I assume that is exbe02)
3. Create exbe03, use SCSI id 7 to add 2 SCSI cards. (that is why exbe02 needs to be shut off!), Whole cluster will share the same quorum, but evs2 will use a seperate disk for evs2!
Add the existing quorum to the first SCSI, and add a new disk to second SCSI. Add an extra network card too.
4. Turn on the exbe03, setup the OS , add it to the exch-cluster.
5. Format the new disk (second SCSI), install exchange (evs2, 192.168.1.10) and use it.
6. Don't turn on the exbe02, that might kill even the whole virtual server environment, setup heartbeat network too.
7. You now have two exchange servers (EVS1 and EVS2), but you can't fail over because you don't have any passive server, don't do this in the real world, that will be really stupid!

You now know how to create evs2, sort of.....

Step by step: How to install exchange 2003 cluster part XII

Simulate a network failure and recovery.
1. On the passive node, go to the "Edit configuration"
2. Change "Local Network" virtual network to "Not connected"
3. The passive node will reports that the nework lost connection.
4. Try to fail over, it should fail, but the active node knows that the passive server is alive.
5. Change "Not connected" to "Local Network" virtual network
6. Change the "HeartBeat" virtual network to "Not connected"
7. Try to fail over, it should work this time.
8. Enable the "HeartBeat" virtual network.

What happened?
Local network is mixed mode, and HeartBeat is dedicated heartbeat network.
The whole purpose of the heartBeat network is to make sure each node knows about others and prevent them from become active when "Local Network" fails.
As you can see, in this lab if both nodes try to be active, that would be a real disaster! and we prevented it.
There is a case you might need only "Local Network" which uses mixed mode. You might need it when you create a campus cluster or geo cluster. Hopefully you have at least 2 ISP and they can back up each other.

Step by step: How to install exchange 2003 cluster part XI

Simulate a node failure and recovery.

1. Delete one of the cluster node and evict the deleted node.
2. Copy and paste base machine and name it same as the deleted node.
3. Set network cards and SCSI shrade disks.
4. Add it as a node and install exchange server.
5. Apply server service packs, exchange service packs and etc.