Wednesday, January 02, 2008

I'm going to jump right into the Entity Framework here, with a simple how-to. If you want a gentle introduction into this technology, you can probably read some future posts, but you will be far better off reading the ADO.Net teamblog or better still, read all of the entries of Danny Simmons.

An OR-mapper needs to be flexible in mapping your domain classes, so you can describe your domain as truthfully as possible. The entity framework positions itself to be more than an OR-Mapper: it sees itself as a translation technology between models. One of those models is defined by your database, the other by your domain (your conceptual model). It's interesting to see what types of inheritance are possible. In this post I will give a sample of the Table Per Type (TPT) inheritance, where you define an inheritance in your conceptual model and use different tables to persist the data.

The way to implement this is fairly straightforward and documented, however, I certainly had some problems getting it to work. As did Scott Allen, so I thought I would create a small walk through for this scenario.

EF_TBT_DBDiagram.jpg

As you can see, my database (SqlExpress) has 3 tables. The 'Person' table, a customer table and an employee table. There are foreign key relations between them, with the Person table being the Primary/Unique Key Base Table and the Customer and Employee tables being the Foreign Key Base Tables. It is important to note that the PersonID column is set up to automatically generate ID's, and the CustomerID and EmployeeID columns are not: we want to create a situation where an object like Customer is persisted to these two tables (customer and person) and the Person table is the one supplying a new ID. The Customer table will just use that PersonID as it's own CustomerID.

That concludes the database schema, next up is the conceptual model, ugh, I mean domain model:

EF_TBT_ConceptualModel.jpg

This is how I want to my conceptual model to look like.  

When you have just generated the model from the database, the foreign key relations are visible. Remove them! Then, also remove the CustomerID and EmployeeID properties that were generated. Add the two inheritance links.
Since Customer already maps to Person (through the inheritance) you do not have to map these again. You do have to add a mapping to the customer table. The key point here is that the CustomerID column needs to be mapped to the PersonID property. Same goes for Employee.

Now, this is all there is to it. Before we dive into the XML generated, let me just point this out: when you update from the database again, the designer will fail. This is a major problem, where the synchronization just isn't good enough. It will encounter problems with the foreign key associations that it has recreated. I hope this gets fixed in the following CTP.

Now, although deceptively easy through the designer, I have a gut-feeling that it's going to be best to actually understand all the XML mapping that is generated. Just like the WPF and WF designers immediately made me turn to learning Xaml and Xoml, I feel that this designer is going to force me to learn CSDL and CS mapping pretty quickly.

When opening the edmx file with the XML editor, quickly use ctrl-E-F to format everything. The designer creates very long lines.
The SSDL content is unimportant, since it represents the database and should not be tinkered with here.
The CSDL is as follows:

    <edmx:ConceptualModels>
      <Schema Namespace="EntityFrameworkTestModel1" Alias="Self" xmlns="http://schemas.microsoft.com/ado/2006/04/edm">
        <EntityContainer Name="EntityFrameworkTestEntities4">
          <EntitySet Name="Person" EntityType="EntityFrameworkTestModel1.Person" />
        </EntityContainer>
        <EntityType Name="Customer" BaseType="EntityFrameworkTestModel1.Person">
          <Property Name="CustomerDiscount" Type="Int32" Nullable="false" />
        </EntityType>
        <EntityType Name="Employee" BaseType="EntityFrameworkTestModel1.Person">
          <Property Name="FunctionName" Type="String" Nullable="false" MaxLength="50" />
        </EntityType>
        <EntityType Name="Person">
          <Key>
            <PropertyRef Name="PersonID" />
          </Key>
          <Property Name="PersonID" Type="Int32" Nullable="false" />
          <Property Name="Firstname" Type="String" Nullable="false" MaxLength="50" />
          <Property Name="Lastname" Type="String" Nullable="false" MaxLength="50" />
        </EntityType>
      </Schema>
    </edmx:ConceptualModels>

Note a container with only one set: the Person set. Then the three types are defined. Only the Person type has a key identified, the Customer and the Employee do not!

The C-S mapping then, glues this model to the SSDL:

      <Mapping Space="C-S" xmlns="urn:schemas-microsoft-com:windows:storage:mapping:CS">
        <EntityContainerMapping StorageEntityContainer="dbo" CdmEntityContainer="EntityFrameworkTestEntities4">
          <EntitySetMapping Name="Person">
            <EntityTypeMapping TypeName="IsTypeOf(EntityFrameworkTestModel1.Person)">
              <MappingFragment StoreEntitySet="Person">
                <ScalarProperty Name="PersonID" ColumnName="PersonID" />
                <ScalarProperty Name="Firstname" ColumnName="Firstname" />
                <ScalarProperty Name="Lastname" ColumnName="Lastname" />
              </MappingFragment>
            </EntityTypeMapping>
            <EntityTypeMapping TypeName="IsTypeOf(EntityFrameworkTestModel1.Customer)">
              <MappingFragment StoreEntitySet="Customer">
                <ScalarProperty Name="PersonID" ColumnName="CustomerID" />
                <ScalarProperty Name="CustomerDiscount" ColumnName="CustomerDiscount" />
              </MappingFragment>
            </EntityTypeMapping>
            <EntityTypeMapping TypeName="IsTypeOf(EntityFrameworkTestModel1.Employee)">
              <MappingFragment StoreEntitySet="Employee">
                <ScalarProperty Name="PersonID" ColumnName="EmployeeID" />
                <ScalarProperty Name="FunctionName" ColumnName="FunctionName" />
              </MappingFragment>
            </EntityTypeMapping>
          </EntitySetMapping>
        </EntityContainerMapping>
      </Mapping>

Here we see the PersonID being mapped to either the CustomerID column or the EmployeeID column. Interesting!

In code, this allows us to do the following:

EntityFrameworkTestEntities4 context = new EntityFrameworkTestEntities4();


// inserting some
Person p = new Person();
p.Firstname = "Ruurd";
p.Lastname = "Boeke";
context.AddToPerson(p);

Customer c = new Customer();
c.Firstname = "Silvia";
c.Lastname = "Banana";
c.CustomerDiscount = 10;
context.AddToPerson(c);

Employee e = new Employee();
e.Firstname = "Ian";
e.Lastname = "Mort";
e.FunctionName = "Developer";
context.AddToPerson(e);

context.SaveChanges();

IQueryable<Person> persons =
from person in context.Person
select person;

foreach (Person person in persons)
{
Console.WriteLine("{0}, {1} is a {2}",
person.Firstname, person.Lastname, person.GetType().Name);
}

Console.ReadLine();

Did you notice that the context was first saved, before the Linq query was executed? It's easy to forget, but the query is executed on the database, not on a Union of objects in the database and in the context.

The output is as expected:

Ruurd, Boeke is a Person
Silvia, Banana is a Customer
Ian, Mort is a Employee

What I really like about this mapping strategy, is the fact that no discriminator column was needed. The system knows that an employee is an employee because of the existence of the record in the employee table. Let's quickly look at the generated query

SELECT
CASE WHEN (( NOT (([UnionAll1].[C2] = 1) AND ([UnionAll1].[C2] IS NOT NULL))) AND ( NOT (([UnionAll1].[C3] = 1) AND ([UnionAll1].[C3] IS NOT NULL)))) THEN '0X' WHEN (([UnionAll1].[C2] = 1) AND ([UnionAll1].[C2] IS NOT NULL)) THEN '0X0X' ELSE '0X1X' END AS [C1],
[Extent1].[PersonID] AS [PersonID],
[Extent1].[Firstname] AS [Firstname],
[Extent1].[Lastname] AS [Lastname],
CASE WHEN (( NOT (([UnionAll1].[C2] = 1) AND ([UnionAll1].[C2] IS NOT NULL))) AND ( NOT (([UnionAll1].[C3] = 1) AND ([UnionAll1].[C3] IS NOT NULL)))) THEN CAST(NULL AS int) WHEN (([UnionAll1].[C2] = 1) AND ([UnionAll1].[C2] IS NOT NULL)) THEN [UnionAll1].[CustomerDiscount] END AS [C2],
CASE WHEN (( NOT (([UnionAll1].[C2] = 1) AND ([UnionAll1].[C2] IS NOT NULL))) AND ( NOT (([UnionAll1].[C3] = 1) AND ([UnionAll1].[C3] IS NOT NULL)))) THEN CAST(NULL AS nvarchar(50)) WHEN (([UnionAll1].[C2] = 1) AND ([UnionAll1].[C2] IS NOT NULL)) THEN CAST(NULL AS nvarchar(50)) ELSE CAST( [UnionAll1].[C1] AS nvarchar(50)) END AS [C3]
FROM [dbo].[Person] AS [Extent1]
LEFT OUTER JOIN (SELECT
    [Extent2].[CustomerID] AS [CustomerID],
    [Extent2].[CustomerDiscount] AS [CustomerDiscount],
    CAST(NULL AS nvarchar(max)) AS [C1],
    cast(1 as bit) AS [C2],
    cast(0 as bit) AS [C3]
    FROM [dbo].[Customer] AS [Extent2]
UNION ALL
    SELECT
    [Extent3].[EmployeeID] AS [EmployeeID],
    CAST(NULL AS int) AS [C1],
    [Extent3].[FunctionName] AS [FunctionName],
    cast(0 as bit) AS [C2],
    cast(1 as bit) AS [C3]
    FROM [dbo].[Employee] AS [Extent3]) AS [UnionAll1] ON [Extent1].[PersonID] = [UnionAll1].[CustomerID]

A left outer join was done with a union of customer and employee. Also note the CASE statements in the main select statement.
I leave it up to you to decide if such SQL statements are okay in your environment. I see no problem with them but I can see them growing pretty fast.

Let's take a look at what happens if you only want your customers. The following Linq expression can be used:

IQueryable<Customer> customers =
from customer in context.Person.OfType<Customer>()
select customer;

When executed, the following SQL is executed:

SELECT
'0X0X' AS [C1],
[Extent1].[CustomerID] AS [CustomerID],
[Extent2].[Firstname] AS [Firstname],
[Extent2].[Lastname] AS [Lastname],
[Extent1].[CustomerDiscount] AS [CustomerDiscount]
FROM [dbo].[Customer] AS [Extent1]
INNER JOIN [dbo].[Person] AS [Extent2] ON [Extent1].[CustomerID] = [Extent2].[PersonID]

All in all very straightforward.

Wednesday, January 02, 2008 6:56:31 PM (Romance Standard Time, UTC+01:00)  #    Comments [5]  |  Trackback
 Sunday, December 30, 2007

I have been on the same project for a long time, and it has given me many opportunities that I am very thankful for:

  • Big time WPF experience. The project was one of the first big WPF projects, starting out when WPF was called Avalon. I have gained a deep insight in the technology and I am very happy to say that I really love it. Yes, it has it's flaws and sometimes feels 'beta-like' still, but it has great potential and is fun to work with.
  • WF experience. I was able to do some great stuff with WF and I view WF as a major step forward for business proces modelling. However, it does have it's downfalls , and I am looking forward to the next version to correct those. Having said that, for specific scenario's, WF is really the way to go, even at this moment.
  • Architecture: I was able to shape the project using these great technologies and I was given a great team of people to implement it with.

However, it's time for a new challenge now! I'm taking some serious time off to relax and look into new technologies. I'm very interested in the Entity Framework. I haven't blogged about these things for a long time, because I was totally into WPF, but I feel a strong desire to start poking at the framework. I also hope to spend some time looking into F# and all the dynamic language stuff that's been hitting us lately (the DLR). Obviously, I'll take a look at Silverlight, potentially the best thing ever to come out of Redmond. I hope it lives up to expectations!

After my little sabbatical, I will be interested in helping out with new projects. Especially if they are working with .net 3.5. Let me know if you have any projects coming up by emailing me.

First up though, u can expect some newbie posts about the Entity Framework!

Best wishes for the new year.

Sunday, December 30, 2007 4:22:31 PM (Romance Standard Time, UTC+01:00)  #    Comments [0]  |  Trackback
 Tuesday, September 11, 2007

Some might remember that I was amongst the first to b*tch about not having a datepicker for wpf. Kevin Moore did his trick, and published a datepicker a long time ago. I'm very happy to see that a rich community is building around wpf. That has led to these two datepickers:

  • Farsi Library FX by Hadi Eskandari (here)
  • The datepicker from wpf contrib by Marlon (here)

Have fun with these!!

Tuesday, September 11, 2007 2:45:50 PM (Romance Standard Time, UTC+01:00)  #    Comments [0]  |  Trackback
 Tuesday, September 04, 2007

Monday morning, our application went live!! Hooray!!

All seems to be going well, although one user had problems when starting the application:

System.Windows.Markup.XamlParseException: Cannot convert string '0,0' in attribute 'StartPoint' to object of type 'System.Windows.Point'. Premature string termination encountered. Error at object 'LinearGradientBrush_1' in markup file 'PresentationUI;V3.0.0.0;31bf3856ad364e35;component/installationprogress.xaml'. ---> System.InvalidOperationException: Premature string termination encountered.

The problem occured even before our application had downloaded. Notice the component/installationprogress.xaml: That's not ours, but belongs to the framework.

I have seen these problems from time to time. We tried to fix it by re-installing the framework. (Sometimes, people have 6.715 instead of 6.920). That didn't work.

In the end, we simply had to discard the users profile. I made a copy of the profile, so maybe I'll find the time to look into the issue in-depth, but I wanted to blog about it, because someone might be running into the same issue.

Tuesday, September 04, 2007 11:18:24 AM (Romance Standard Time, UTC+01:00)  #    Comments [2]  |  Trackback
 Monday, September 03, 2007

This is turning into a hassle. I must confess that I feel that Microsoft does not have a good story on this one!

When thinking about versioning within the realm of workflow, there are a few things you have to know:

  • You will need to use strong signing for your processes, the activities, the External Data Exchange services and the items you put on the queue (we use these to correlate commands to queues, bypassing the weirdness of correlation in WF)
  • What is persisted to the datastore is a blob. That blob is created using serialization surrogates and use the normal binary serialization format. However, because of the surrogates, it is difficult (although not impossible) to touch your workflow instance directly, instead of going through the runtime. The surrogates are there for a reason: the serialization process of a workflowinstance is not a straight-forward process: all the activity contexts have to be serialized as well, as do the dependency properties etc.
  • The blob does not only persist your fields, but persists the complete structure of your running instance, called a template. So all the activities (initialized or not) are in that template.
  • Timers and their delays are persisted in a separate list by the surrogate. So, if your workflow instance is in a delay with 9 days left, this information is written in a timerCollectionList, with a guid pointing to the delay (remember, that delay is instantiated in a particular activityContext). It is not simple to correlate these. They are the main problem when you wish to just update your process.

Microsoft does not offer a smart way to upgrade version 1.0 to 2.0 of your workflow instance. When you have version 1.0 in your database, and make one little change to your process, dehydration will not work because of an index-out-of-bounds exception: remember that the persisted blob has the full template of the instance. So when you changed your process and added or removed an activity anywhere, the dehydration process is trying to map the persisted template to a type in your assembly and fails because of the different activity tree.

Therefor, you can do two things:

  1. Run both assemblies Side by Side
  2. Use workflow changes to change your current version to a 2.0 compatible version. 

Let's start by discussing option 2. Say you have created version 2.0 of your instance and try to rehydrate. Since you strongnamed, the runtime will throw an exception because it can not find your old assembly. You can place an assembly redirect in your config, telling the clr to try to use version 2.0 assemblies to instantiate your 1.0 blob. This will then fail because of that changed structure of your template.
The solution at hand is to use workflow changes to get in there, and change the structure of that 1.0 template to match that of a 2.0 version. You can of course only do this by loading in the old assembly side-by-side, but now you only have to do that once during an update-batch. After that, your normal application is able to use your 2.0 assembly to instantiate your 1.0 (but now structurally modified) instance.

The problem here, is that you have to build big workflow change scripts. I have not yet seen someone automate that (do a diff on the templates and generate the workflow changes). If that were available and rock-stable, this might be a good strategy to take. Until then, it's way too much work. (Let me know if this turns out to be super-simple!)

Option 1 is bad as well. Sure, loading in your old assemblies is possible. But what Microsoft forgot is that I want to change my external data exchange service as well (if only in version number) and the objects that I put in my queue. Since your old 1.0 process is expecting a 1.0 service to talk to, or 1.0 version commands, it will not be able to communicate!! This can be mitigated by adding the 1.0 External service to the runtime when loading the 1.0 assemblies, and maybe only using bcl types on the queues, but it's really a shame to have to do that. Certainly when you have processes that last 5 years, and you have 20 versions to keep up with.....

My advise is to really try to understand the way your application will use workflow. For us, I was able to make these assumptions:

  • There are a few states that need to be monitored by a delay activity of say 20 days.. When after that 20 days our process has not moved out of that state, something needs to happen.
  • Most states do not need that. Therefor, I can actually bring the process to a completed state. That actually is not what I prefer, however, since the state of a process can be derived from my domain objects, I can always construct the process at will and bring it in the correct state (with the new version!!). By completing the processes whenever possible, I will have a much smaller amount of processes to deal with in the datastore.
  • Most importantly: I found out that after processing an external event, the process will always return back into a state. It will never start a delay of more then a few minutes within a sequence. So I can guarantee that my workflow, when persisted, is not waiting inside a while-loop or whatever. All long delays are the first child within an eventdriven activity.

The last point reduces our problem big time, because it would be nearly impossible to build an update for a workflow instance that is waiting in the middle of a sequence. Basically, that is what we will be doing in our project: Build an update batch, that will load version 1.0 instances, kill them, create 2.0 version instances and write back to the database with the same guid.

The steps to build an update batch are:

  1. use workflow tracking or something to write the version of your process to your datastore. We have an oracle persistence layer. When we build it, we constructed a new column 'type' in the database and write the fullname of the type in there (which includes version number).
  2. load your old assembly so you can instantiate the blob
  3. instantiate the blob using reflection to directly get access to your instance
  4. do an export of your fields and other stuff you need. You know your processes intimately, so this should not be a problem
  5. delete the row from the datastore  (remember to start a transaction!)
  6. create a new type, using the runtime and the guid of the old instance
  7. call an import event or whatever, that the process will use to bring itself to the correct state
  8. persist

The hard part are the delays. Basically, you can find the list of timers using reflection. However, it is cumbersome to correlate the guids to the correct delay activities. My solution would be the following: during state changes within your process, keep a dictionary of the statename and the moment (Datetime) you transitioned to it. When importing, use this list and the delay.timeoutdurationEvent to setup your timer: normally, it would be DateTime.Now.Add(timeoutlenght). This time, you will use the original DateTime, and your delay activity will not have been 'reset'.

It's not pretty, and it will be necessary to put constraints on your processes. But it might work just fine for you! Let me know..

Monday, September 03, 2007 6:59:38 PM (Romance Standard Time, UTC+01:00)  #    Comments [4]  |  Trackback
 Friday, August 31, 2007

As we are scrambling to get our application ready for it's end-users, we are starting to notice some troublesome behavior by PresentationHost.exe: it tends to go up to 99% CPU-utilization and refuses to go down again.
Obviously we have noticed PresentationHost going up very fast during our debugging sessions, but what goes up, always does go down again. However, now I'm receiving lots of reports from our end-users that claim the 'application is very slow'. This turns out to be that rogue process, which refuses to go down!!

When we were able to attach windbg to it, we did not find a clear cause. It wasn't executing any of our code, to be sure. But it's definitely busy (no, there aren't any running animations!).

We've also seen multiple instances of the process, where we'd only expect one.

This is turning into a real headache. When we figure it out, I'll be sure to post!

Friday, August 31, 2007 8:24:14 PM (Romance Standard Time, UTC+01:00)  #    Comments [0]  |  Trackback
 Tuesday, August 28, 2007

The new entityframework ctp was released. The official statement is here.
It includes the designer, so I'm very curious how this fares against nHibernate! Congratz to the team!

The Devguy does a summary of the features. Read this hilarious piece:

  • In previous CTPs the underlying provider connection was opened when the ObjectContext was constructed and held open for the life of the context—this would create issues, for instance, with databases where licensing was based on the number of concurrently open connections since the connection might be held open for an extended period of time—even when the connection is not being used.  The new model keeps the connection closed as much as possible while still appropriately dealing with transactions and avoiding promotion of those transactions from local to DTC where possible.

Really?

I mean, so, they now no longer keep the connection open all the time and instead have opted to close it when it's not in use..... because of licensing issues??

ROFL. I would think there are better reasons not to keep a connection open for too long, but that's just me.

;-)

Tuesday, August 28, 2007 1:23:20 PM (Romance Standard Time, UTC+01:00)  #    Comments [4]  |  Trackback
 Thursday, August 02, 2007

Just ran across a comment from Daniel Puzey here, with the following excellent advise:

Often, by default, you'll get an error reported at Line 1 of the xaml, which is an obvious lie. You can catch the original exception, though:

- Open the "Exceptions" window (Debug/Exceptions) in Visual Studio.
- Click "add"
- Add "System.Windows.Markup.XamlParseException"
- Check the box to break on throw for this exception.
- Hit F5!

You'll find that the XamlParseException you catch is much more descriptive, and will give the correct position in the xaml file.

Thursday, August 02, 2007 6:44:01 PM (Romance Standard Time, UTC+01:00)  #    Comments [0]  |  Trackback