Monday, April 07, 2008
Disconnected N-Tier objects using EntityFramework

I have just checked in a sample application that uses a server to retrieve data with EF and a client that is totally unaware of EF. Changes on the client are changetracked and serialized in a pretty xml format. Then, the server attaches to the graph and is able to use a context to save changes.
In this post I will explain some of the parts in more detail. I hope to follow up on some point with a screencast. First, let's take a step back and look at the problem (skip this if you've read my blog lately).

Problems with N-Tier EF

Normal Entity Framework usage will generate classes for you that are adorned with [DataMember] attributes, making them serializable using WCF's DataContractSerializer.
The class that was generated has intimate knowledge of Entity Framework, which presents itself as EF-specific tags in the XML: not pretty and not SOA at all!

Further, on the receiving end (the client that deserializes these objects) those same classes need to be around to deserialize into. This means that you will have to use Entity Framework on the client: not SOA at all. This pretty much limits the usage of your domain objects to a .Net client. (I'm not sure if Silverlight will be able to use EF. If not: you will not be able to use these objects in Silverlight as well)

I probably pointed to this post before, but Daniel Simmons talks about using EF in your architecture. He acknowledges the problem but does not see it as a very big issue. In some situations, this might not be a big problem, but in others it is.

Then, there is the issue of serializing the objects with all their relations in-tact. It's harder than you would hope. Daniel again has a solution, that you can find here. He basically creates a serializable class (EntityBag) which groups all relations and objects together and knows how to reassemble them. This way you can use your objects on the client using EF and have a pretty seamless experience. Great!! However, it's not going to work for other client technologies, so forget about SOA.

Also: I prefer to work in a more 'agile'/'domain driven' way. Having a layer manifest itself all the way up to the client, where it does not belong, really kills flexibility.

Solution scope

My aim is to provide a solution that will allow me to:

  1. Work with POCO classes (as persistence ignorant as I can get)
  2. No EF references on the client
  3. Full changetracking on the client that can be attached easily
  4. Use the same Domain classes (with it's logic and validation rules) on both server and client: so no duplication of code

We should be able to work with our classes in Silverlight even.

The Parts of the Solution

I have partitioned the solution into some smaller projects. All use PostSharp to do it's magic, but you will not notice a thing about it. (If you are unaware: PostSharp will perform a step after compilation to transform your IL. This way, no source code pollution).

POCO  (only on server):

  1. this project will enrich your domain objects with EF information, so there is no need to litter your objects with EF specific. Feel the agile love.
  2. has the ability to provide 'original values' to the serializer
  3. has the ability/knowledge on how to attach a graph to an EF context

Circular serializer (both on server as client):

  1. creates surrogate objects that will be stand-ins during serialization. This fixes the circular serialization problem.
  2. also creates datamembers that will hold your original values during serialization
  3. creates a method (CopyFromSurrogate) that allows the serializer to fill the original object with values from the surrogate.

The circular serializer does more then just solve the circular serialization problem. I might refactor this to make it clearer.

Editable Business Object (only on client):

  1. this project implements IEditableObject (and INotifyPropertyChanged) on your domain object
  2. knows how to keep 'original values' and provide them to the serializer

Again, this project might be split.

 

So there are 3 different parts here. Let's look at that with a picture:

image

Basically, the circular serializer is the glue that is used on both the server and the client. It knows how to serialize your objects and asks for help to get to the original values. On the client, the EditableBusinessObject can provide that, on the server, it is the original POCO project.

Using them is dead simple, as I'll show.

Look at some code

I've used the School database and let EF generate the entity mappings. I did not change anything from the defaults, which basically means that the classes we end up with are pretty awful. I don't care about them though for this sample.

image

I then set the generated code to 'not compile' and created the following beautiful classes by hand:

  0 namespace Domain
  1 {
  2 [DataContract(Name = "Person", Namespace = "tstNS")]
  3  public class Person
  4 {
  5   [DataMember]
  6   public int PersonID { get; set; }
  7   [DataMember]
  8   public string LastName { get; set; }
  9   [DataMember]
  10   public string FirstName { get; set; }
  11   [DataMember]
  12   public DateTime? HireDate { get; set; }
  13   [DataMember]
  14   public DateTime? EnrollmentDate { get; set; }
  15
  16   [DataMember]
  17   public ICollection<Enrollment> Enrollments { get; set; }
  18   [DataMember]
  19   public ICollection<Course> Courses { get; set; }
  20
  21 }
  22
  23 [DataContract(Name = "Course", Namespace = "tstNS")]
  24  public class Course
  25 {
  26   [DataMember]
  27   public int CourseID { get; set; }
  28   [DataMember]
  29   public string Title { get; set; }
  30   [DataMember]
  31   public string Days { get; set; }
  32   [DataMember]
  33   public DateTime Time { get; set; }
  34   [DataMember]
  35   public string Location { get; set; }
  36   [DataMember]
  37   public int Credits { get; set; }
  38   [DataMember]
  39   public Department Department { get; set; }
  40
  41   [DataMember]
  42   public ICollection<Enrollment> Enrollments { get; set; }
  43   [DataMember]
  44   public ICollection<Person> Persons { get; set; }
  45
  46 }
  47 [DataContract(Name="Department", Namespace="tstNS")]
  48  public class Department
  49 {
  50   [DataMember]
  51   public int DepartmentID { get; set; }
  52   [DataMember]
  53   public string Name { get; set; }
  54   [DataMember]
  55   public Decimal Budget { get; set; }
  56   [DataMember]
  57   public DateTime StartDate { get; set; }
  58   [DataMember]
  59   public int Administrator { get; set; }
  60
  61   [DataMember]
  62   public ICollection<Course> Courses { get; set; }
  63
  64 }
  65 [DataContract(Name = "Enrollment", Namespace = "tstNS")]
  66  public class Enrollment
  67 {
  68   [DataMember]
  69   public int EnrollmentID { get; set; }
  70   [DataMember]
  71   public Decimal? Grade { get; set; }
  72
  73   [DataMember]
  74   public Course Course { get; set; }
  75   [DataMember]
  76   public Person Person { get; set; }
  77
  78 }
  79 }
  80

Now, I need a trick to get seperate versions of this on the client and the server. I altered the msbuild script to add a conditional compile symbol that is the name of the solution. You could go fancy with this technique, by matching the build name and stuff. I just used this:

  <PropertyGroup>



    <DefineConstants>$(DefineConstants);$(SolutionName)</DefineConstants>


  </PropertyGroup>

(Please let me know when someone uses this technique to target both Silverlight and regular .net. It's possible!)
So, now it's time to apply our postsharp attributes. I did that in the assembly info file:

#if Server
[assembly: Poco("SchoolEntitiesConnection", AttributeTargetAssemblies = "Domain", AttributeTargetTypes = "Domain.*")]
[assembly: CreateSerializeSurrogate("EntityFrameworkContrib.PostSharp4EF.ISerializationHelp, EntityFrameworkContrib.PostSharp4EF", AttributeTargetAssemblies = "Domain", AttributeTargetTypes = "Domain.*")]
#endif
#if Client
[assembly: EditableBusinessObjectAttribute(AttributeTargetAssemblies="Domain", AttributeTargetTypes="Domain.*")]
[assembly: CreateSerializeSurrogate("EditableBusinessObjects.Postsharp.Public.IEditableBusinessObject, EditableBusinessObjects.Postsharp.Public", AttributeTargetAssemblies="Domain", AttributeTargetTypes="Domain.*")]
#endif

Pretty neat, isn't it? This notifies postsharp to either use Poco or EditableBusinessObject to modify the compiled code. I only have one place to maintain this code.
As you can see, the circular serialize (CreateSerializeSurrogate) takes the name of a type that it can use to get 'original value' information from.

I also created an interface project (just a nice best-practice) to identify my operation contract:

  0 [CircularReferenceSurrogateAttribute]
  1 [ServiceContract(Namespace="Http://sitechno.School")]
  2  public interface ISchoolService
  3 {
  4   [OperationContract]
  5   Person GetPerson(int id, bool fetchEnrollments, bool fetchCourses);
  6
  7   [OperationContract]
  8   void SavePerson(Person person);
  9
  10   [OperationContract]
  11   IList<Course> GetCourses();
  12
  13   [OperationContract]
  14   IList<Department> GetDepartments();
  15
  16 }

See the CircularReferenceSurrogateAttribute on line 0: it applies a datacontract surrogate manager, that will substitute surrogate types for the real types during serialization.

Now, for the server:

  0  public class SchoolService : ISchoolService
  1 {
  2   public Person GetPerson(int id, bool fetchEnrollments, bool fetchCourses)
  3   {
  4    using (SchoolContext context = new SchoolContext())
  5    {
  6     Person p = (from person in context.Person
  7        where person.PersonID  == id
  8           select person).First();
  9
  10     if (fetchEnrollments)
  11     {
  12      ((IRelationshipLoader)p).Load("Enrollments");
  13      foreach (Enrollment r in p.Enrollments)
  14      {
  15       ((IRelationshipLoader)r).Load("Course");
  16       ((IRelationshipLoader)r).Load("Person");
  17      }
  18     }
  19    
  20     if (fetchCourses)
  21     {
  22      ((IRelationshipLoader)p).Load("Courses");
  23      foreach (Course c in p.Courses)
  24      {
  25       ((IRelationshipLoader)c).Load("Department");
  26       ((IRelationshipLoader)c).Load("Enrollments");
  27       ((IRelationshipLoader)c).Load("Persons");
  28
  29      }
  30     }
  31
  32     ((IContextAware)p).CreateSerializableState(context);
  33
  34     return p;
  35    }
  36
  37   }
  38
  39
  40   public void SavePerson(Person person)
  41   {
  42    using (SchoolContext context = new SchoolContext())
  43    {
  44     ((IContextAware)person).AttachGraphToContext(context, entity => entity.GetType().Name);
  45
  46     context.SaveChanges();
  47    }
  48   }
  49
  50
  51   public IList<Course> GetCourses()
  52   {
  53    using (SchoolContext context = new SchoolContext())
  54    {
  55     List<Course> courses = context.Course.ToList();
  56     courses.ForEach(crs => ((IContextAware)crs).CreateSerializableState(context));
  57     return courses;
  58
  59    }
  60   }
  61
  62   public IList<Department> GetDepartments()
  63   {
  64    using (SchoolContext context = new SchoolContext())
  65    {
  66     List<Department> deps = context.Department.ToList();
  67     deps.ForEach(dp => ((IContextAware)dp).CreateSerializableState(context));
  68     return deps;
  69    }
  70   }
  71
  72 }

Before someone tells me that I'm not doing the includes/spans correctly in the GetPerson method: I'll do it better when I do a more complete sample.

As you can see, for methods that return classes, I call the CreateSerializableState method. Methods that accept a graph have to call the AttachGraphToContext method. Obviously, these can and should be context extensibility methods.

  • The CreateSerializableState method will use the context to get the 'original' values and load them into your object. Your class actually has fields to hold this information (as created by Poco).
  • The AttachGraphToContext method is quite complex and will look at a graph and it's original values. It will first set the original values, than do an acceptchanges and change only those properties that were actually changed. It also fixes up relationships, using add/attach/delete.

The client:

  0   static void Main(string[] args)
  1   {
  2    SchoolService svc = new SchoolService();
  3
  4    IList<Department> departments = svc.GetDepartments();
  5
  6    Person Fadi = svc.GetPerson(4, false, false);
  7
  8    Console.WriteLine(String.Format("Working on {0} {1}, Hired at {2:D}",
  9     Fadi.FirstName, Fadi.LastName, Fadi.HireDate));
  10    Console.WriteLine();
  11
  12    Console.WriteLine("The ID is not generated by the database in this instance, so please give me a number that is not used already");
  13    int id = Int32.Parse(Console.ReadLine());
  14
  15    Console.WriteLine("I can bug you for a title as well, can't I?");
  16    string title = Console.ReadLine();
  17
  18    Course newCourse = new Course { CourseID = id, Credits = 10, Days = "MT", Time = DateTime.Now, Title = title };
  19    newCourse.Persons = new List<Person> { Fadi }; // normally, your object ctor would have done this
  20    newCourse.Enrollments = new List<Enrollment>();
  21    newCourse.Department = departments.First();
  22    newCourse.Department.Courses.Add(newCourse);
  23
  24    Fadi.FirstName = "Fadi,Changed at " + DateTime.Now.ToShortTimeString();
  25
  26
  27    Fadi.Courses.Add(newCourse);
  28
  29
  30    svc.SavePerson(Fadi);
  31
  32    Console.WriteLine("Finished, go check your database. I added the new course to department:" + newCourse.Department.Name);
  33    Console.ReadLine();
  34
  35
  36   }

One important thing to know is that on the client, there is no longer a concept of relations, only lists. So you will need to hook up classes from both angles when you connect them: see lines 21 and 22.

Things to note
  • There is a LOT going on, but you are shielded from it and can work in a blissful persistence ignorance way!
  • Your objects will implement quite a few interfaces that you can use if you'd like
  • The client side implements INotifyPropertyChanged (and raises events correctly) but the ICollection implementations are still just Lists<>. I will change that to ObservableCollections for you WPF lovers! (which includes me)
  • I have not implemented deleting yet. So you can unhook a relationship, but that does not delete the object. I don't think that it should either! I will probably implement some 'MarkAsDelete' method on the client-side
  • There is a strong optimization for speed. I, for instance, generate a method 'SetValue(string propname, object val)' and corresponding GetValue so that the framework does not need to use reflection or dynamic method generation. Still, I haven't covered everything yet, so there is still some expensive reflection going on. The overall goal is certainly to reduce this to null.
  • This is not a version 1.0 release. Do not treat it as such. It's complex matter, and some things are not supported yet. Still 98% is finished.
  • I rushed the samples. There is a wpf sample that has the worst codebehind that exists. Do not open it, your eyes will hurt. When I have time I'll build a proper MVC sample that actually doesn't crash.

I am very excited to have gotten this far. Since PostSharp supports Silverlight, I'll try to get this to work for silverlight as well, so we can build some serious RIA's! I'll try to follow up with a screen cast soon.

kick it on DotNetKicks.com

 |  | 
Monday, April 07, 2008 9:25:31 PM (Romance Standard Time, UTC+01:00)  #    Comments [2]  |  Trackback
 Friday, March 21, 2008
Progress on light objects on the client, full blown EF entities on the backend

I'm working hard on bringing a couple of my projects together that will enable domainobjects on the client without Entity Framework references. I thought I'd give a small update on how that's working out for me.

The endresult

What I want is one visual studio project: 'Domain' that holds business objects for me that I can use on both the client and the server. What I do not want, is to have to think about EntityFramework contexts on the client. Since I do not persist my objects on the client, I see no reason to actually create a context on the client, just to be able to do changetracking. Nor do I want to see any EF specific plumbing in the messages between that client and server: I lose interoperability if I do that. Also, if my client is Silverlight (for instance) I might not want to bring the EF assemblies over (if that's even possible?).

So, the endresult should be an easy way to serialize and deserialize objects on the client and server, and a way to build up the object correctly on the server so it can be attached to the context again.

Please check Daniel's EntityBag sample for a very sophisticated way of serializing your context. He does go down the route of serializing the objectcontext, which does mean you need the entityframework on the client. Read about his motivations for doing this here.

Let's see it

I'll first show some code and then walk you through it.

  0    using (OneSimpleType.OneSimpleTypeConnection context = new OneSimpleType.OneSimpleTypeConnection())
  1    {
  2     // clear out database
  3     foreach (OneSimpleType.Person old in context.Person)
  4     {
  5      context.DeleteObject(old);
  6     }
  7     context.SaveChanges();
  8    }
  9
  10    OneSimpleType.Person p = new OneSimpleType.Person { Firstname = "Ruurd", Lastname = "Boeke" };
  11    IEditableBusinessObject ep = p as IEditableBusinessObject;
  12    string MsgOnWire = "";
  13
  14    using (OneSimpleType.OneSimpleTypeConnection context = new OneSimpleType.OneSimpleTypeConnection())
  15    {
  16     context.AddToPerson(p);
  17     context.SaveChanges();  // at this point, there should be 'original values'
  18
  19     // the server is changing the object without yet saving
  20     p.Firstname = "ServerChanged";
  21
  22     // now we need to create a version of the object that can be serialized well
  23     p = (OneSimpleType.Person)context.CreateSerializableVersion(p);
  24
  25     DataContractSerializer s = new DataContractSerializer(p.GetType(), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  26
  27     MsgOnWire = GetWellFormedToContract(p, s);
  28
  29    }
  30
  31    // imagine we are the client getting a message and deserializing it
  32    DataContractSerializer s2 = new DataContractSerializer(typeof(OneSimpleType.Person), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  33    StringReader reader = new StringReader(MsgOnWire);
  34    XmlReader xreader = XmlReader.Create(reader);
  35    OneSimpleType.Person p2 = (OneSimpleType.Person)s2.ReadObject(xreader);
  36
  37    // edit something on the client
  38    p2.Lastname = "ClientChanged";
  39
  40
  41    // we have edited everything, let's get back to the server
  42    s2 = new DataContractSerializer(typeof(OneSimpleType.Person), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  43    MsgOnWire = GetWellFormedToContract(p2, s2);
  44
  45    // we're at the server
  46    using (OneSimpleType.OneSimpleTypeConnection context = new OneSimpleType.OneSimpleTypeConnection())
  47    {
  48     DataContractSerializer s3 = new DataContractSerializer(typeof(OneSimpleType.Person), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  49     StringReader reader3 = new StringReader(MsgOnWire);
  50     XmlReader xreader3 = XmlReader.Create(reader3);
  51     OneSimpleType.Person p3 = (OneSimpleType.Person)s3.ReadObject(xreader3);
  52
  53     context.AttachedDeserializedVersion(p3, "Person");
  54
  55     context.SaveChanges();
  56    }
  57

We start out on the serverside.
On line 17, we have just saved an object to the database. 
We continue using that (attached, and thus changetracked) object and change a property (just for the fun of it).
On line 27 we serialize it like you've seen me do in previous posts.

On line 32, we pretend to be on the client, which uses the message to deserialize to a compatible type. Note, that this type does not need to be 'IPoco' enhanced. It does need to be enhanced for serialization and editing.

We change a property 'on the client' on line 38. Notice that this object does not have a context attached at all.
At line 43 we have once again serialized this new object and on line 46 we pretend to be on the server again.

There we create a new context and deserialize our message to an object. Then we call an extension method of context to attach this version to the context. Note that I have to pass the setname (working on somehow getting to this information differently).
Finally we save.

Only the changed properties are flagged as dirty, so efficient SQL can be generated by EF.

The extension method is shown here:

  0   public static void AttachedDeserializedVersion(this ObjectContext context, object source, string setname)
  1   {
  2    // TODO: how to get setname from metadata or something
  3
  4    Debug.Assert(source is IEntityWithKey);
  5    Debug.Assert(source is IEditableBusinessObject);
  6
  7    IEditableBusinessObject eS = (IEditableBusinessObject)source;
  8
  9    // we go to the loaded state
  10    eS.SetReadMode(ReadMode.Loaded);
  11
  14    string fullEntitySetName = context.DefaultContainerName + "." + setname;
  15
  16    EntityKey createdKey = context.GetEntityKey(fullEntitySetName, source);
  17    ((IEntityWithKey)source).EntityKey = createdKey;
  18
  19    // attach this version
  20    context.Attach((IEntityWithKey)source);
  21
  22    eS.ReplayLoadedToNormal();
  23
  24    eS.SetReadMode(ReadMode.Normal);
  25
  26
  27   }

You see I have quite a bit of functionality in the IEditableBusinessObject interface.

The deserialized version of the object has a 'loaded' state (or better: original state) and a 'current' state. Before attaching at line 20, I first set my object to be in 'original' mode. Then I have a replay method that will compare the current values to the loaded versions and touch the setters of the properties of the changed versions. This notifies the changetracker of EF to flag a property as dirty.

This all works fine, but both code and api are pretty crude. I have no complextype or relationshiptype support, but that will come, if I don't run into major problems along the way.

I hope you are as excited about this as I am. Let me know!

 |  | 
Friday, March 21, 2008 6:21:28 PM (Romance Standard Time, UTC+01:00)  #    Comments [2]  |  Trackback
 Monday, March 17, 2008
Sneak preview: serializing objects with original values

I'll probably dedicate a bigger post to this soon, but I wanted to show you a domainmodel, some code and the xml it generates.
What you see here is two attributes that take on quite a bit of work.
EditableBusinessObject implements IEditableObject for you and also allows you to copy the currentvalues to a loadedvalues state.
CreateSerializeSurrogate generates a surrogate class that knows how to deal with loadedvalues and with circular references.
Together, they form the backbone of the client-side of your domainmodel.

Here is an example.

DomainModel:
(notice this is the full class, no other properties are here)

  0 [EditableBusinessObject]
  1 [CreateSerializeSurrogate]
  2 [Serializable]
  3 [DataContract(Namespace = "myNamespace", Name = "Person")]
  4  public class Person
  5 {
  6   [DataMember]
  7   public int IntProperty { get; set; }
  8
  9   [DataMember]
  10   public string StringProperty { get; set; }
  11
  12   [DataMember]
  13   public string StringProperty2 { get; set; }
  14
  15   int neverSetInt;
  16   [DataMember]
  17   public int NeverSetInt
  18   {
  19    get
  20    {
  21     return neverSetInt;
  22    }
  23    set { neverSetInt = value; }
  24   }
  25
  26   [DataMember]
  27   public List<string> StringLijst { get; set; }
  28
  29   [DataMember]
  30   public List<int> IntLijst { get; set; }
  31
  32   public Person()
  33   {
  34    StringLijst = new List<string>();
  35    IntLijst = new List<int>();
  36   }
  37
  38 }
  39

Some testcode:
(notice line 16 where we do a endEdit. If we had cancelled, we would've had a proper rollback)

  0    Person p = new Person();
  1    p.IntProperty = 1;
  2    p.StringProperty = "Ruurd";
  3    p.StringProperty2 = "Boeke";
  4
  5    p.StringLijst.Add("a");
  6    p.IntLijst.Add(1);
  7
  8    (p as IEditableBusinessObject).CopyCurrentToLoaded();
  9
  10    (p as IEditableBusinessObject).BeginEdit();
  11
  12    p.IntLijst.Add(2);
  13    p.StringLijst.Add("b");
  14    p.StringProperty = "Ruurd Boeke";
  15
  16    (p as IEditableBusinessObject).EndEdit();
  17
  18    DataContractSerializer s = new DataContractSerializer(p.GetType(), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  19
  20    string outMessage = GetWellFormedToContract(p, s);

And the generated xml:
(notice how the lists and 'StringProperty' are changed, and see the original value).

<PersonSurrogate xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="myNamespace">



    <IntLijst xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">



        <d2p1:int>1</d2p1:int>



        <d2p1:int>2</d2p1:int>



    <