Monday, 07 April 2008

I have just checked in a sample application that uses a server to retrieve data with EF and a client that is totally unaware of EF. Changes on the client are changetracked and serialized in a pretty xml format. Then, the server attaches to the graph and is able to use a context to save changes.
In this post I will explain some of the parts in more detail. I hope to follow up on some point with a screencast. First, let's take a step back and look at the problem (skip this if you've read my blog lately).

Problems with N-Tier EF

Normal Entity Framework usage will generate classes for you that are adorned with [DataMember] attributes, making them serializable using WCF's DataContractSerializer.
The class that was generated has intimate knowledge of Entity Framework, which presents itself as EF-specific tags in the XML: not pretty and not SOA at all!

Further, on the receiving end (the client that deserializes these objects) those same classes need to be around to deserialize into. This means that you will have to use Entity Framework on the client: not SOA at all. This pretty much limits the usage of your domain objects to a .Net client. (I'm not sure if Silverlight will be able to use EF. If not: you will not be able to use these objects in Silverlight as well)

I probably pointed to this post before, but Daniel Simmons talks about using EF in your architecture. He acknowledges the problem but does not see it as a very big issue. In some situations, this might not be a big problem, but in others it is.

Then, there is the issue of serializing the objects with all their relations in-tact. It's harder than you would hope. Daniel again has a solution, that you can find here. He basically creates a serializable class (EntityBag) which groups all relations and objects together and knows how to reassemble them. This way you can use your objects on the client using EF and have a pretty seamless experience. Great!! However, it's not going to work for other client technologies, so forget about SOA.

Also: I prefer to work in a more 'agile'/'domain driven' way. Having a layer manifest itself all the way up to the client, where it does not belong, really kills flexibility.

Solution scope

My aim is to provide a solution that will allow me to:

  1. Work with POCO classes (as persistence ignorant as I can get)
  2. No EF references on the client
  3. Full changetracking on the client that can be attached easily
  4. Use the same Domain classes (with it's logic and validation rules) on both server and client: so no duplication of code

We should be able to work with our classes in Silverlight even.

The Parts of the Solution

I have partitioned the solution into some smaller projects. All use PostSharp to do it's magic, but you will not notice a thing about it. (If you are unaware: PostSharp will perform a step after compilation to transform your IL. This way, no source code pollution).

POCO  (only on server):

  1. this project will enrich your domain objects with EF information, so there is no need to litter your objects with EF specific. Feel the agile love.
  2. has the ability to provide 'original values' to the serializer
  3. has the ability/knowledge on how to attach a graph to an EF context

Circular serializer (both on server as client):

  1. creates surrogate objects that will be stand-ins during serialization. This fixes the circular serialization problem.
  2. also creates datamembers that will hold your original values during serialization
  3. creates a method (CopyFromSurrogate) that allows the serializer to fill the original object with values from the surrogate.

The circular serializer does more then just solve the circular serialization problem. I might refactor this to make it clearer.

Editable Business Object (only on client):

  1. this project implements IEditableObject (and INotifyPropertyChanged) on your domain object
  2. knows how to keep 'original values' and provide them to the serializer

Again, this project might be split.


So there are 3 different parts here. Let's look at that with a picture:


Basically, the circular serializer is the glue that is used on both the server and the client. It knows how to serialize your objects and asks for help to get to the original values. On the client, the EditableBusinessObject can provide that, on the server, it is the original POCO project.

Using them is dead simple, as I'll show.

Look at some code

I've used the School database and let EF generate the entity mappings. I did not change anything from the defaults, which basically means that the classes we end up with are pretty awful. I don't care about them though for this sample.


I then set the generated code to 'not compile' and created the following beautiful classes by hand:

  0 namespace Domain
  1 {
  2 [DataContract(Name = "Person", Namespace = "tstNS")]
  3  public class Person
  4 {
  5   [DataMember]
  6   public int PersonID { get; set; }
  7   [DataMember]
  8   public string LastName { get; set; }
  9   [DataMember]
  10   public string FirstName { get; set; }
  11   [DataMember]
  12   public DateTime? HireDate { get; set; }
  13   [DataMember]
  14   public DateTime? EnrollmentDate { get; set; }
  16   [DataMember]
  17   public ICollection<Enrollment> Enrollments { get; set; }
  18   [DataMember]
  19   public ICollection<Course> Courses { get; set; }
  21 }
  23 [DataContract(Name = "Course", Namespace = "tstNS")]
  24  public class Course
  25 {
  26   [DataMember]
  27   public int CourseID { get; set; }
  28   [DataMember]
  29   public string Title { get; set; }
  30   [DataMember]
  31   public string Days { get; set; }
  32   [DataMember]
  33   public DateTime Time { get; set; }
  34   [DataMember]
  35   public string Location { get; set; }
  36   [DataMember]
  37   public int Credits { get; set; }
  38   [DataMember]
  39   public Department Department { get; set; }
  41   [DataMember]
  42   public ICollection<Enrollment> Enrollments { get; set; }
  43   [DataMember]
  44   public ICollection<Person> Persons { get; set; }
  46 }
  47 [DataContract(Name="Department", Namespace="tstNS")]
  48  public class Department
  49 {
  50   [DataMember]
  51   public int DepartmentID { get; set; }
  52   [DataMember]
  53   public string Name { get; set; }
  54   [DataMember]
  55   public Decimal Budget { get; set; }
  56   [DataMember]
  57   public DateTime StartDate { get; set; }
  58   [DataMember]
  59   public int Administrator { get; set; }
  61   [DataMember]
  62   public ICollection<Course> Courses { get; set; }
  64 }
  65 [DataContract(Name = "Enrollment", Namespace = "tstNS")]
  66  public class Enrollment
  67 {
  68   [DataMember]
  69   public int EnrollmentID { get; set; }
  70   [DataMember]
  71   public Decimal? Grade { get; set; }
  73   [DataMember]
  74   public Course Course { get; set; }
  75   [DataMember]
  76   public Person Person { get; set; }
  78 }
  79 }

Now, I need a trick to get seperate versions of this on the client and the server. I altered the msbuild script to add a conditional compile symbol that is the name of the solution. You could go fancy with this technique, by matching the build name and stuff. I just used this:


(Please let me know when someone uses this technique to target both Silverlight and regular .net. It's possible!)
So, now it's time to apply our postsharp attributes. I did that in the assembly info file:

#if Server
[assembly: Poco("SchoolEntitiesConnection", AttributeTargetAssemblies = "Domain", AttributeTargetTypes = "Domain.*")]
[assembly: CreateSerializeSurrogate("EntityFrameworkContrib.PostSharp4EF.ISerializationHelp, EntityFrameworkContrib.PostSharp4EF", AttributeTargetAssemblies = "Domain", AttributeTargetTypes = "Domain.*")]
#if Client
[assembly: EditableBusinessObjectAttribute(AttributeTargetAssemblies="Domain", AttributeTargetTypes="Domain.*")]
[assembly: CreateSerializeSurrogate("EditableBusinessObjects.Postsharp.Public.IEditableBusinessObject, EditableBusinessObjects.Postsharp.Public", AttributeTargetAssemblies="Domain", AttributeTargetTypes="Domain.*")]

Pretty neat, isn't it? This notifies postsharp to either use Poco or EditableBusinessObject to modify the compiled code. I only have one place to maintain this code.
As you can see, the circular serialize (CreateSerializeSurrogate) takes the name of a type that it can use to get 'original value' information from.

I also created an interface project (just a nice best-practice) to identify my operation contract:

  0 [CircularReferenceSurrogateAttribute]
  1 [ServiceContract(Namespace="Http://sitechno.School")]
  2  public interface ISchoolService
  3 {
  4   [OperationContract]
  5   Person GetPerson(int id, bool fetchEnrollments, bool fetchCourses);
  7   [OperationContract]
  8   void SavePerson(Person person);
  10   [OperationContract]
  11   IList<Course> GetCourses();
  13   [OperationContract]
  14   IList<Department> GetDepartments();
  16 }

See the CircularReferenceSurrogateAttribute on line 0: it applies a datacontract surrogate manager, that will substitute surrogate types for the real types during serialization.

Now, for the server:

  0  public class SchoolService : ISchoolService
  1 {
  2   public Person GetPerson(int id, bool fetchEnrollments, bool fetchCourses)
  3   {
  4    using (SchoolContext context = new SchoolContext())
  5    {
  6     Person p = (from person in context.Person
  7        where person.PersonID  == id
  8           select person).First();
  10     if (fetchEnrollments)
  11     {
  12      ((IRelationshipLoader)p).Load("Enrollments");
  13      foreach (Enrollment r in p.Enrollments)
  14      {
  15       ((IRelationshipLoader)r).Load("Course");
  16       ((IRelationshipLoader)r).Load("Person");
  17      }
  18     }
  20     if (fetchCourses)
  21     {
  22      ((IRelationshipLoader)p).Load("Courses");
  23      foreach (Course c in p.Courses)
  24      {
  25       ((IRelationshipLoader)c).Load("Department");
  26       ((IRelationshipLoader)c).Load("Enrollments");
  27       ((IRelationshipLoader)c).Load("Persons");
  29      }
  30     }
  32     ((IContextAware)p).CreateSerializableState(context);
  34     return p;
  35    }
  37   }
  40   public void SavePerson(Person person)
  41   {
  42    using (SchoolContext context = new SchoolContext())
  43    {
  44     ((IContextAware)person).AttachGraphToContext(context, entity => entity.GetType().Name);
  46     context.SaveChanges();
  47    }
  48   }
  51   public IList<Course> GetCourses()
  52   {
  53    using (SchoolContext context = new SchoolContext())
  54    {
  55     List<Course> courses = context.Course.ToList();
  56     courses.ForEach(crs => ((IContextAware)crs).CreateSerializableState(context));
  57     return courses;
  59    }
  60   }
  62   public IList<Department> GetDepartments()
  63   {
  64    using (SchoolContext context = new SchoolContext())
  65    {
  66     List<Department> deps = context.Department.ToList();
  67     deps.ForEach(dp => ((IContextAware)dp).CreateSerializableState(context));
  68     return deps;
  69    }
  70   }
  72 }

Before someone tells me that I'm not doing the includes/spans correctly in the GetPerson method: I'll do it better when I do a more complete sample.

As you can see, for methods that return classes, I call the CreateSerializableState method. Methods that accept a graph have to call the AttachGraphToContext method. Obviously, these can and should be context extensibility methods.

  • The CreateSerializableState method will use the context to get the 'original' values and load them into your object. Your class actually has fields to hold this information (as created by Poco).
  • The AttachGraphToContext method is quite complex and will look at a graph and it's original values. It will first set the original values, than do an acceptchanges and change only those properties that were actually changed. It also fixes up relationships, using add/attach/delete.

The client:

  0   static void Main(string[] args)
  1   {
  2    SchoolService svc = new SchoolService();
  4    IList<Department> departments = svc.GetDepartments();
  6    Person Fadi = svc.GetPerson(4, false, false);
  8    Console.WriteLine(String.Format("Working on {0} {1}, Hired at {2:D}",
  9     Fadi.FirstName, Fadi.LastName, Fadi.HireDate));
  10    Console.WriteLine();
  12    Console.WriteLine("The ID is not generated by the database in this instance, so please give me a number that is not used already");
  13    int id = Int32.Parse(Console.ReadLine());
  15    Console.WriteLine("I can bug you for a title as well, can't I?");
  16    string title = Console.ReadLine();
  18    Course newCourse = new Course { CourseID = id, Credits = 10, Days = "MT", Time = DateTime.Now, Title = title };
  19    newCourse.Persons = new List<Person> { Fadi }; // normally, your object ctor would have done this
  20    newCourse.Enrollments = new List<Enrollment>();
  21    newCourse.Department = departments.First();
  22    newCourse.Department.Courses.Add(newCourse);
  24    Fadi.FirstName = "Fadi,Changed at " + DateTime.Now.ToShortTimeString();
  27    Fadi.Courses.Add(newCourse);
  30    svc.SavePerson(Fadi);
  32    Console.WriteLine("Finished, go check your database. I added the new course to department:" + newCourse.Department.Name);
  33    Console.ReadLine();
  36   }

One important thing to know is that on the client, there is no longer a concept of relations, only lists. So you will need to hook up classes from both angles when you connect them: see lines 21 and 22.

Things to note
  • There is a LOT going on, but you are shielded from it and can work in a blissful persistence ignorance way!
  • Your objects will implement quite a few interfaces that you can use if you'd like
  • The client side implements INotifyPropertyChanged (and raises events correctly) but the ICollection implementations are still just Lists<>. I will change that to ObservableCollections for you WPF lovers! (which includes me)
  • I have not implemented deleting yet. So you can unhook a relationship, but that does not delete the object. I don't think that it should either! I will probably implement some 'MarkAsDelete' method on the client-side
  • There is a strong optimization for speed. I, for instance, generate a method 'SetValue(string propname, object val)' and corresponding GetValue so that the framework does not need to use reflection or dynamic method generation. Still, I haven't covered everything yet, so there is still some expensive reflection going on. The overall goal is certainly to reduce this to null.
  • This is not a version 1.0 release. Do not treat it as such. It's complex matter, and some things are not supported yet. Still 98% is finished.
  • I rushed the samples. There is a wpf sample that has the worst codebehind that exists. Do not open it, your eyes will hurt. When I have time I'll build a proper MVC sample that actually doesn't crash.

I am very excited to have gotten this far. Since PostSharp supports Silverlight, I'll try to get this to work for silverlight as well, so we can build some serious RIA's! I'll try to follow up with a screen cast soon.

kick it on

Monday, 07 April 2008 21:25:31 (Romance Standard Time, UTC+01:00)  #    Comments [5]  |  Trackback
 Wednesday, 02 April 2008

I have had my problems with the EF designer for some time now, but Noam Ben-Ami talks about the improvements that are being made in the next drop here.

They seem to have fixed a lot of issues with the update-from-database option, which should make it a lot more useful. Cool!

Wednesday, 02 April 2008 10:07:46 (Romance Standard Time, UTC+01:00)  #    Comments [0]  |  Trackback
 Thursday, 27 March 2008

[important: this post belongs to a whole host of other posts. Basically it's about my efforts of building a light-weight n-tier disconnected solution for Entity Framework]

Another status update. I've worked on connecting up a clean graph to a EntityFramework context and found it far from straight-forward! I ended up with very little code though, which is a good sign.

The approach of not serializing the object context, makes it harder to figure out when an object was added or removed. On the client, I keep copies of the 'original' collection and I compare the new collection with the old collection. Problem there is that it's quite possible that an object was just removed from a collection and added to another collection. How do we know we have to do a real 'delete' on it?
I have not yet figured this out ;-)

Here is some code for you to look at:

  0    Customer c = new Customer { Name = "Ruurd Boeke" };
  1    Car car1 = new Car { Make = "Saab", Customer = c };
  2    Order order1 = new Order { Amount = 2, Customer = c };
  5    IEditableBusinessObject eC = c as IEditableBusinessObject;
  6    string MsgOnWire = "";
  8    // at this point, we are not at all attached
  9    using (SimpleRelationshipTestEntities context = new SimpleRelationshipTestEntities())
  10    {
  11     context.AddToCustomer(c);
  12     context.SaveChanges();  // this instructs context to start changetracking
  14     ((IContextAware)c).CreateSerializableState(context);
  16     DataContractSerializer s = new DataContractSerializer(c.GetType(), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  18     MsgOnWire = s.GetWellFormedToContract(c);
  19    }

The message on the wire looks like this:

<CustomerSurrogate xmlns:i="" xmlns="myNamespace">
    <Name>Ruurd Boeke</Name>
    <OriginalValue_Name>Ruurd Boeke</OriginalValue_Name>

Next step is to receive this xml on the client, deserialize it and change some stuff:

  0    DataContractSerializer s2 = new DataContractSerializer(typeof(CustomerClient), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  1    CustomerClient c2 = (CustomerClient)s2.UnwrapFromString(MsgOnWire);
  3    // delete
  4    c2.Cars.First().Customer = null;
  5    c2.Cars.Clear();
  7    // add
  8    OrderClient order2 = new OrderClient { Amount = 100, Customer = c2 };
  9    c2.Orders.Add(order2);
  11    // change
  12    c2.Name = "Changed";
  14    // we have edited everything, let's get back to the server
  15    DataContractSerializer s3 = new DataContractSerializer(typeof(CustomerClient), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  16    MsgOnWire = s3.GetWellFormedToContract(c2);

This client is removing the cars. Note that we do not get the automatic hookup of relations that we used to get on the server: there is no EF Relationship manager on the client!

<CustomerSurrogate xmlns:i="" xmlns="myNamespace">
    <Cars />
    <OriginalValue_Name>Ruurd Boeke</OriginalValue_Name>

We see that there are no current cars, but the original value of cars still has them. That is how I figure out to delete the cars on the server.
Did you also notice how the new order does not have a ID assigned? The server will take care of it.

  0    // we're at the server
  1    using (SimpleRelationshipTestEntities context = new SimpleRelationshipTestEntities())
  2    {
  3     DataContractSerializer s4 = new DataContractSerializer(typeof(Customer), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  4     Customer c3 = (Customer)s4.UnwrapFromString(MsgOnWire);
  6     ((IContextAware)c3).AttachGraphToContext(context,
  7      delegate(object source)
  8      {
  9       return source.GetType().Name;   // our setnames correspond to our classname.
  10      });
  13     context.SaveChanges();
  14    }

Now, we're deserializing and attaching the graph to the context. The delete on line 7 is a way of returning entitysetnames. EF needs to know the entitysetname to be able to attach an object to a context, but there is no way for me to know it. So at this point I let you make the decision yourself.

The database is correctly filled and we are happy.

Now, I really want to check in this source, but there is a lot to be done still. The API I have created is monstrous.
Also, I think there are quite a few situations where this will break horribly.

Once I get just a little happier about it, I will do a screencast building a server using Entity Framework and a silverlight client that is editing.

Thursday, 27 March 2008 17:20:28 (Romance Standard Time, UTC+01:00)  #    Comments [10]  |  Trackback
 Monday, 24 March 2008

Found through Julie's blog: a new tool was released on Code Gallery from the EF Team, called EF Mapping Helper.

It will help you play around with mappings and see what the result would be in the corresponding CSDL, SSDL and MSL generated xml.

Read more on Julie's blog and quickly download the tool!

Monday, 24 March 2008 17:08:14 (Romance Standard Time, UTC+01:00)  #    Comments [12]  |  Trackback