Visual Studio 2005 SP1 Released

Get it here!

Explicit Impersonation and Async Thread Identity in ASP.NET

I've been doing some asynchronous thread programming in our server components and last night ran into a gotcha that took me a few hours to figure out so I thought I'd post this one to save people some trouble.

I've implemented a Fire and Forget asynchronous pattern similar to Mike Woodring's sample so that I could execute longer running processes than the typical web request on a spearate thread and return back to the client immediately. These processes connect to a SQL database using windows integrated security.

All was working just dandy on my development XP box running without impersonation. In this scenario the components run under the default ASPNET identity and connect to a local database. (In Windows 2003/IIS6 the default identity is NETWORK SERVICE.) However when I went to deploy it on our testing servers I ran into a problem with the asynchronous thread identities. They were throwing exceptions trying to connect to the database.

Our test rig is set up as two Windows 2003 servers, one app server and one database server on their own little domain. We set the app server's web.config with explicit impersonation of a least priveledged domain account that is windows authenticated to the database. All works fine for client request threads, however the identity of the async threads were that of the application pool, not the explicit domain user and was therefore causing problems connecting to the database. I figured I could change the application pool identity, but I wasn't satisfied with having to remember another configuration setting. I really wanted the Web.config to be the only place for this and I was perplexed as to why the main thread's identity wasn't getting propagated.

I still am not sure as to why this is the case since the documentation for ThreadPool.QueueUserWorkItem makes it seem like this should work in .NET 2.0. I ended up augmenting the AsynHelper class to impersonate a WindowsIdentity. Here's the code and it's usage (comments/suggestions welcome!):

Imports System.Threading
Imports System.Security

Friend Class AsyncHelper

    Private Shared wc As New WaitCallback(AddressOf CallMethod)

    Public Shared Sub FireAndForget(ByVal d As [Delegate], _
            ByVal wi As Principal.WindowsIdentity, _
            ByVal ParamArray args As Object())

        ThreadPool.QueueUserWorkItem(wc, New TargetInfo(d, args, wi))
    End Sub

    Private Shared Sub CallMethod(ByVal o As Object)
        Dim ti As TargetInfo = DirectCast(o, TargetInfo)

        'This is necessary so this thread impersonates the 
        'calling thread's identity. This is important when 
        'running under ASP.NET explicit impersonation.
        ti.Identity.Impersonate()

        'Invoke the method, passing the arguments
        ti.Target.DynamicInvoke(ti.Args)
    End Sub

    Private Class TargetInfo
        Private m_target As [Delegate]
        Private m_args As Object()
        Private m_wi As Principal.WindowsIdentity

        ReadOnly Property Target() As [Delegate]
            Get
                Return m_target
            End Get
        End Property

        ReadOnly Property Args() As Object()
            Get
                Return m_args
            End Get
        End Property

        ReadOnly Property Identity() As Principal.WindowsIdentity
            Get
                Return m_wi
            End Get
        End Property

        Sub New(ByVal d As [Delegate], _
            ByVal args As Object(), _
            ByVal wi As Principal.WindowsIdentity)

            m_target = d
            m_args = args
            m_wi = wi
        End Sub
    End Class

End Class

And here's a usage example:

Private Delegate Sub ExecuteQueryDelegate(ByVal personID As Integer) 

Public Sub BeginFetch(ByVal personID As Integer)
    Dim dlgt As New ExecuteQueryDelegate(AddressOf ExecuteQuery)

    ' Initiate the asynchronous call.
    AsyncHelper.FireAndForget(dlgt, Principal.WindowsIdentity.GetCurrent(), personID)
End Sub

'This method runs on an asynchronous thread
Private Sub ExecuteQuery(ByVal personID As Integer) 
    Dim resultSet As DataTable = Me.LongRunningProcesses()
    Me.SaveResults(resultSet)
End Sub

Linq's the Word

Many thanks to all those who came out on Thursday to play with the latest Linq CTP at the East Bay.NET User's Group in Pleasanton! Here's the link to all the presentation code.

The presentation went really well and the samples sparked a lot of discussion and interest with the group who had never seen language integrated query. Most of the attendees were very excited and anxious for it to release, as am I.

Linq changes the way you think about solving problems in your code. That coupled with Visual Basic's dynamic interfaces and XML literals you definately have the most flexible language I have ever seen. Read here for an overview of these features coming in the next version.

Bay.NET UG presentation code

Many thanks to all those who came out on Tuesday to see me speak at the Bay.NET User's Group in San Fransisco! Those of you who missed it, I did a more in-depth presentation of the Data Sources and Data Binding in VS2005 session that I gave at DevTeach. Here's the link to all the presentation code. Have Fun!

Debugging Windows Services in Visual Studio

Last week I started writing a Windows Service in Visual Studio for some build and test automation programs. What I have always found annoying (but understandable) about windows services is the debugging. Because your code is set up to run as a service when you select the Windows Service template in VS that means you have to install the service, start it, and then attach to the process in the debugger instead of being able to click the "start debugging" button on the debugger toolbar like I'm used to with other project types.

Previously, I followed the advice in this article by creating a setup program that easily allowed me to install and uninstall the service directly from the Solution Explorer in VS. However I still have to either call System.Diagnostics.Debugger.Break() in my code (and remember to remove it when I'm done!) or attach to the process in order to debug it.

There's an easier way. If you look in the Main entry point of the service you will see how the service is loaded. (In C# this is in the Program.cs file that's created for you and in VB.NET this code resides in the service's Designer.vb file.) You can modify this code by adding a DEBUG compiler directive to control how the process starts:

    <MTAThread()> _
    Shared Sub Main()
#If DEBUG Then
        ' Start the process as a non-service for debugging only.
        ' Stop the debugger to stop the process.

        Dim service As New MyService 
        service.Execute() 

        System.Threading.Thread.Sleep(System.Threading.Timeout.Infinite)
#Else
        Dim ServicesToRun() As System.ServiceProcess.ServiceBase

        ' More than one NT Service may run within the same process. To add
        ' another service to this process, change the following line to
        ' create a second service object. For example,
        '
        '   ServicesToRun = New System.ServiceProcess.ServiceBase () {New Service1, New MySecondUserService}
        '
        ServicesToRun = New System.ServiceProcess.ServiceBase() {New MyService}

        System.ServiceProcess.ServiceBase.Run(ServicesToRun)
#End If
    End Sub

Now when your configuration is set to debug, you can hit F5 to debug like normal. Stop the debugger to stop the process. Keep in mind that the OnStart and OnStop methods of your service will not run so you should break your functionality into a callable method on your service class (in this example, I called it "Execute()"). Have fun!

Programming Computers

Today I spoke to some of my mom's computer students about programming at the school where she teaches. What a change from last week where I'm teaching other developers advanced programming topics at DevTeach. Today I'm encouraging 14 year olds to get into the field and demo-ing Visual Basic 2005 Express and the starter kits. We had a blast poking around the black jack starter kit.

I started the presentation by introducing myself and telling them the kind of software products we write at GiftRAP Corporation. I told them that I worked at Microsoft a couple years ago on the Visual FoxPro team in the developer division. Then I explained what a Microsoft MVP is and that I still get to contribute ideas to Microsoft even though I don't work there.

Since I wanted to encourage them to start programming, I went through some of my history as a kid learning to program. I showed them my first computer, the Atari 400, and my first programming language; it was 1980 and I was 8 years old. I got some bright eyes in a couple kids when I said that. I got some gasps when I told them it retailed for $595.00. I realize now that my parents really sacrificed to buy that computer for me; they didn't spend that type of money back then. Then I skipped to 1986, the year I was in 8th grade and showed them the Amiga 2000 HD. I think HD meant that you got a hard drive, something I didn't have on my Franklin "IBM PC clone". I wanted to draw all day when I was a young teen but I loved computers, so I begged my mom for an Amiga; the best in computer graphics at the time. And as a bonus, it came with an 8086 motherboard so I could run MS-BASIC. With 1MB RAM I was cooking with gas. Finally I showed them a picture of a few of the machines I have today including my 2 year old SmartPhone. I waved it in the air and told them it had 32 times more memory than my Amiga 2000. A couple kids laughed and a couple wanted me to cut to the chase -- let's program already!

I opened up VS and selected the card game starter kit template and we started tweaking it. I changed the card skins to a snoopy face and made Player 1 a picture of the president. They laughed, they stared, they asked questions. I'd say they seemed pretty darn interested (for teenagers). I pointed them to a VS Express Edition game programming site that looked pretty fun. I think I proved to them that programming is not only a scientific process but also a creative one. Mom said to save the presentation and she wants me back earlier in the year next year. I'm really glad I had the opportunity to do this with the kids today, it's almost harder to do these types of lessons than the professional sessions I do. It's cool, I think some things I was saying to them sunk in and hopefully will make a tiny impression on their future career choices. It was a very rewarding talk for me.

Here's the links I gave them. This is VB Express free edition:
http://msdn.microsoft.com/vstudio/express/vb/
These are the starter kits:
http://msdn.microsoft.com/vstudio/express/vb/starterkit/default.aspx
Here's the sweepstakes and more games:
http://www.upgradeyourgame.com/
Links to beginer videos and learning links:
http://msdn.microsoft.com/vstudio/express/vb/next/default.aspx
http://msdn.microsoft.com/vstudio/express/vb/learning/default.aspx
Here's the community forum:
http://forums.microsoft.com/msdn/showforum.aspx?forumid=24&siteid=1

"Best and Worst of .NET" Interviews at DevTeach

Mario Cardinal interviewed the speakers at DevTeach on what we thought were the best and worst parts of Microsoft .NET. Mario snuck up on me and put me on the spot. If he asked me again today I would probably have thought of completely different things to say. There's just so much I like ;-). Check it out here!

Amazing Countrysides

So now that DevTeach is over, Alan and I are in beautiful Vermont staying with Julie Lerman and her husband Rich. What a great feel this countryside has, rolling green hills and birch trees everywhere. Living in the East Bay Area of California we have lots of rolling green hills but the grass is very different; much longer, and dies in the summer. We also have oak trees, not these white wood birch so it's so cool to see a different countryside. (Although all this birch is a little Blair Witch-y for me at night ;-P) It's my first time in Vermont but I'm sure I'll be back.

Wonderful Montreal in May

DevTeach is over; what a great time! Montreal was absolutely beautiful. No rain, nice and warm, in the 70s. I couldn't have asked for anything more. My sessions went very well and I got great feedback from the attendees on the conference as a whole. For session coverage check out the UT. The focus was on Data and Alan gave a great keynote on LINQ. If you haven't done it already, download the latest CTP and start playing with it now. It really will change the way you program... everything.

Another MVP year!

Yahooo! I was awarded Microsoft Solutions Architecture MVP again this year. Thanks to everyone involved over at Microsoft and to all the readers of my blog in this awesome community. Hope to see you all next month in Montreal!

The best definition of software architecture I've heard

My friend and fellow Architect MVP, Eric Newcomer mentioned in a blog post something I said when we were having dinner last week in SF and it ended up in SOA Web Services Journal. I'm glad all my ranting pays off once in a while ;-).

I'm speaking at DevTeach 2006

I'm speaking again this year at DevTeach 2006 in Montreal May 8th-12th.

I'm doing three sessions this year, one called "VS2005 Data Sources and Data Binding" and a two part session on "Designing Applications for Multiple Smart Client UIs". That one also demonstrates a server architecture that supports windows clients as well as mobile devices against the same centralized data. We'll also discuss occasionally connected scenarios and architecture options we have today and in the future. You can see all the speakers and sessions listed here. Hope you can make it!

Allowing impersonated accounts to write to your custom event log

Here's a little tid-bit of information that I've had laying around for a while that I'd thought I'd share. .NET lets us easily create our own custom event logs with:

EventLog.CreateEventSource(source, logname)

As long as your an administrator running this code, it will create a custom event log. On a Windows 2003 machine only interactive users, services, batch accounts, admins, and server operators can read and write to this log. If you have components impersonating a least privileged user account then you will not be able to write to your event log. There's a couple ways I know of that you can do to fix this. You can allow any authenticated user write access to your log or you can allow the specific user SID write access. Open up the registry editor to your event log CustomSD key:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Eventlog\MyLogName\CustomSD

To allow all authenticated users write access add the following security permission:

(A;;0x0002;;;AU)

To allow a specific user write access you would specify their SID:

(A;;0x0002;;;SID-OF-USER-ACCOUNT)

For more information see Development Impacts of Security Changes in Windows Server 2003

Connecting to SQL-Server Express on a remote XP SP2 machine

If you are experiencing problems connecting to a SQL-Server Express database on an XP SP2 remote box via Remote Desktop - there's a HotFix here you can get to resolve the issue.

New text rendering in .NET 2.0 WinForms

Winforms 2.0 added support for drawing GDI text with the new TextRenderer class available in the Systems.Windows.Forms namespace. Using this class to draw strings is now the preferred way instead of using the previous Graphics.DrawString method. So not to break previous WinForms applications, there is a new property on controls called UseCompatibleTextRendering, and if set to True this will draw text as it used to in 1.1.

On application startup, you can globally set this on all controls with:

Application.SetCompatibleTextRenderingDefault(False)

This draws all the labels in the system much clearer than previous versions especially when using the Tahoma 8.25pt font. When using this font at design time and dragging fields from your datasources window, the labels will line up using the new 2.0 behavior, but if you forget to set this at runtime using the above code, you may notice your labels not lining up properly.

If drawing strings manually, like from paint handlers on controls, you'll want to use the new TextRenderer.

So instead of:

Graphics.DrawString(someText, Me.Font, New SolidBrush(Me.ForeColor), rect)

Use this code instead:

TextRenderer.DrawText(Graphics, someText, Me.Font, rect, Me.ForeColor, TextFormatFlags.Left)

ActiveNick is coming to townnnnnnnn...

...he knows when your phone is sleeping, he know when your PDA is awake, he knows what's bad and good about the Compact Framework, so come to his sessions for goodness sake! Oh, you better watch out...

Yep, that's right, Nick Landry (a.k.a. ActiveNick) is coming to San Francisco on March 8th to speak at the Bay.NET User group.

Nick is a good friend of mine and a very dynamic speaker. He'll be giving us a crash course on developing mobility solutions so come on out and see us if you're in town!

Binary Serialization of DataSets in .NET 2.0

Last week I mentioned that I would post some test results on the different options we now have with serializing DataSets; Binary (new in 2.0), XML, and custom serialization surrogate. First, let me direct your attention to an updated surrogate class that I used in the test that can handle the new DateTimeMode property on columns.

Now before you read these numbers I'd like to be clear about the scenarios I tested. To make it easy on me I set up the internet test to work with one of our current product's test beds already set up with some real test data. There are two servers in this test bed, one is the application server hosting remote components in IIS (HttpChannel) and accessed by the client using the BinaryFormatter. The second server is the database server. So what I was measuring was also including the time it took to select the records from the database through the components (no business logic, just filling an untyped dataset from a stored proc simple select -- about 19 columns with various data types). The servers are in Florida and I work from home in California where I was running the client. I was accessing them over a cable modem. In the local test I had the exact same code and databse running in all tiers on my local development machine to simulate no network latency.

That said, we don't want look at the numbers per se, look at the trend -- that is what is important here. There are also a lot of factors that affect serialization over the internet on a cable modem and you'll notice that there are a few anomalies in the local numbers with small sets of data, probably because my dev machine hiccupped at that moment.

So to recap, this is what we're measuring;

1) The time it takes to make the remote call to activate the component,
2) The time it takes to select the records from the database and create the dataset,
3) In the case of the surrogate class, the time it takes to convert the dataset to a series of byte arrays,
4) The time it takes to transmit the data,
5) In the case of the surrogate class, the time it takes to convert the byte arrays back into a dataset.

My conclusion is that native Binary serialization of datasets is only better over a long wire when the number of rows are in the thousands. Our application is 99% data entry forms so we would never be returning that much data. The surrogate class does have a slight overhead if your network is not congested/not the internet, however nothing that the user would notice. Therefore, for now, I'm sticking with the surrogate class for our application. I'll let you know if I change my mind later based on more formal load testing.

Okay here are the performance numbers:

LOCAL TEST (Low Network Latency)
# Records	Size (bytes)			Transmission Time (ms)
	Surrogate	Binary	XML	Surrogate	Binary	XML
2	13,761	56,198	11,673	15.63	31.25	15.63
10	15,261	57,419	15,754	15.63	31.25	15.63
20	17,133	58,972	21,018	31.25	31.25	15.63
100	32,233	71,626	63,434	31.25	31.25	31.25
500	107,336	134,641	276,350	62.50	46.88	62.50
1000	203,943	216,285	550,832	93.75	78.13	109.38
2000	392,296	374,665	1,087,187	171.88	125.00	250.00
4000	772,451	694,776	2,174,403	343.75	265.63	515.63
8000	1,521,110	1,323,693	4,383,707	734.38	531.25	968.75
INTERNET TEST (High Network Latency)
# Records	Size (bytes)			Transmission Time (ms)
	Surrogate	Binary	XML	Surrogate	Binary	XML
2	13,761	56,198	11,673	312.50	578.13	312.50
10	15,261	57,419	15,754	328.13	640.63	390.63
20	17,133	58,972	21,018	343.75	655.43	343.75
100	32,233	71,626	63,434	421.88	671.88	593.75
500	107,336	134,641	276,350	906.25	1078.13	1875.00
1000	203,943	216,285	550,832	1484.38	1562.50	3531.25
2000	392,296	374,665	1,087,187	2640.63	2515.63	6750.00
4000	772,451	694,776	2,174,403	5312.50	4500.00	13312.50
8000	1,521,110	1,323,693	4,383,707	9687.50	8406.25	26609.38

I LOVE SQL-EXPRESS

One of the key features of our products is the ability to support multiple deployment scenarios without recompilation. Scenarios ranging from desktop single user to few user workgroup to client-server to web-based n-tier with the option of NLB. This is all done with configuration files. All of our deployments up to now have been web-based n-tier against SQL-Server 2005. So yesterday the owner of the company asks me to load a single user demo system on his laptop. No problem, this should be fun and a good test of SQL-Express.

I didn't realize how easy this was going to be. First thing was to load SQL-Server Express edition on his machine (which also loads .NET Fx 2.0). Since he wanted some demo data, I needed to grab the database on our test DB server. I fired up SSMS, connected to the the test server, detached the database, took a copy the physical Mdf file, then reattached the database. I placed the copy of the database file in the same folder with all our application files and then changed the application connection string to:

Data Source=.\SQLEXPRESS;AttachDbFilename=C:\MyAppFolder\DBNAME_Data.mdf;Integrated Security=True;User Instance=True

This sets up a single user file-based connection string. That's it! You don't have to perform an attach or run any sql scripts if you just want single user file-based access. Wow. I love it! This is a REALLY REALLY REALLY nice feature of SQL-Server. This same database file can be attached to a SQL-Server 2005 database server as the scalabiltiy needs of our customers change. Added with our own application deployment options, our product provides solutions for a very broad customer base. Good job Microsoft!

Serializing data across time zones in .NET 2.0

This week I've been implementing support in our framework for the new DateTimeMode setting on the DataColumn class. This property is used on DateTime columns to determine a couple things; 1) it specifies how dates should be stored in the column and 2) how a date should be serialized when marshalling across time zones. Setting the DateTimeMode to Local or Utc will not only affect serialization but will also affect how the data is stored in the column. Setting to Unspecified or UnspecifiedLocal does not affect the storage, just the serialization. In .NET 1.x the dataset would always serialize dates as UnspecifiedLocal. This means it would apply the appropriate local time offset when it deserialized. So if you entered the value 12/1/2005 00:00:00 in Florida and passed that dataset to California it would show up as 11/30/2005 21:00:00. This can be a problem depending on the meaning of the datetime value. If you are say storing this in a database and/or not using the time part, when you display the date part to the user you have a problem.

One way to solve this in 1.x is to use a surrogate class which also provides the added ability to serialize the data as true binary. When wrapping the dates up in the surrogate before serialization, you can have it not apply the offset to the DateTime columns. However this approach works across the entire set of columns in the dataset; there's no granularity.

So here comes the DateTimeMode in .NET 2.0. You can set each individual column to the mode you want and the serialization will behave exactly how you want. Great. So lets start designing our typed datasets and you'll notice a DateTimeMode property in the property sheet in the designer. Set that to the mode you want and regenerate your typed datasets... right? WRONG. There's a bug in the designer (and XSD.EXE) where it refuses to code spit the DateTimeMode even though it's properly declared in the xsd file. Well that totally sucks. For more info see the bug report.

So the work around is that you have to set the DateTimeMode at runtime before serializing. Also be careful if your merging an untyped dataset into a typed dataset because the DateTimeMode on the typed dataset will prevail in that case. (See the documentation on compatible merges.)

So with all the workaround code I now have running I turn my attention to the new binary serialization support in the 2.0 DataSet. My initial performance numbers are NOT impressive at all. In fact, the Xml serialization is much faster for small sets of data. This is in drastic contrast to the surrogate class I was using in 1.x. The surrogate class performs better in all cases. I'm updating the surrogate to include the new DateTimeMode support and once I have it all tested I'll post a new GotDotNet sample. I'll also post some more formal numbers once I finish my tests.

Happy New Year and all that jazz....

HAPPY NEW YEAR!

Okay I know it's a little late. Sorry I haven't posted in a long while but lots of things were going on for me during the holidays.

For one, I was heads down for the last month upgrading our framework to take advantage of .NET 2.0 and Visual Studio 2005. Things like generics, custom events, ADO.NET enhancements, new WinForms data binding stuff to mention a few. I've also been playing around with the Visual Studio Team System Tester load test tools. Love it.

Also, I was busy in December prepping for my first Christmas dinner. Thank god for my sister. She's a Martha Stewart Wonder Woman. I couldn't have done it without her. She's a scientist (a real one, not computers) and she had the entire Christmas prep and dinner planned out in an Excel spreadsheet. The roast leaving the oven was T-minus zero and everything before the roast was negative minutes and everything after the roast was positive numbers. Of course, at time of execution we ended up drinking too much champagne and the whole planning spreadsheet went out the window and the "Force" took over. We did great though; even Nona said it was a wonderful dinner. I love cooking but I need to watch more Emeril.

Finally, Alan and I threw a NYE party and it was off the hook (do people say that anymore?). Anyways, it was a lot of fun. All the neighbors came and we all got drunk on ice wine martinis my sister was shaking. Some got a little too drunk. Luckily I'm a beer drinker and maintained my cool (and my dinner unlike some people who used my bathroom). Only two broken glasses and no one in the hospital so it was a success.

Resolutions this year? Blog more, read more, get an Xbox 360 and set up media center, get new furnaces, and find a landscaper before the weeds outside grow to "Land of the Lost" size this spring.