Monday, 07 April 2008

I have just checked in a sample application that uses a server to retrieve data with EF and a client that is totally unaware of EF. Changes on the client are changetracked and serialized in a pretty xml format. Then, the server attaches to the graph and is able to use a context to save changes.
In this post I will explain some of the parts in more detail. I hope to follow up on some point with a screencast. First, let's take a step back and look at the problem (skip this if you've read my blog lately).

Problems with N-Tier EF

Normal Entity Framework usage will generate classes for you that are adorned with [DataMember] attributes, making them serializable using WCF's DataContractSerializer.
The class that was generated has intimate knowledge of Entity Framework, which presents itself as EF-specific tags in the XML: not pretty and not SOA at all!

Further, on the receiving end (the client that deserializes these objects) those same classes need to be around to deserialize into. This means that you will have to use Entity Framework on the client: not SOA at all. This pretty much limits the usage of your domain objects to a .Net client. (I'm not sure if Silverlight will be able to use EF. If not: you will not be able to use these objects in Silverlight as well)

I probably pointed to this post before, but Daniel Simmons talks about using EF in your architecture. He acknowledges the problem but does not see it as a very big issue. In some situations, this might not be a big problem, but in others it is.

Then, there is the issue of serializing the objects with all their relations in-tact. It's harder than you would hope. Daniel again has a solution, that you can find here. He basically creates a serializable class (EntityBag) which groups all relations and objects together and knows how to reassemble them. This way you can use your objects on the client using EF and have a pretty seamless experience. Great!! However, it's not going to work for other client technologies, so forget about SOA.

Also: I prefer to work in a more 'agile'/'domain driven' way. Having a layer manifest itself all the way up to the client, where it does not belong, really kills flexibility.

Solution scope

My aim is to provide a solution that will allow me to:

  1. Work with POCO classes (as persistence ignorant as I can get)
  2. No EF references on the client
  3. Full changetracking on the client that can be attached easily
  4. Use the same Domain classes (with it's logic and validation rules) on both server and client: so no duplication of code

We should be able to work with our classes in Silverlight even.

The Parts of the Solution

I have partitioned the solution into some smaller projects. All use PostSharp to do it's magic, but you will not notice a thing about it. (If you are unaware: PostSharp will perform a step after compilation to transform your IL. This way, no source code pollution).

POCO  (only on server):

  1. this project will enrich your domain objects with EF information, so there is no need to litter your objects with EF specific. Feel the agile love.
  2. has the ability to provide 'original values' to the serializer
  3. has the ability/knowledge on how to attach a graph to an EF context

Circular serializer (both on server as client):

  1. creates surrogate objects that will be stand-ins during serialization. This fixes the circular serialization problem.
  2. also creates datamembers that will hold your original values during serialization
  3. creates a method (CopyFromSurrogate) that allows the serializer to fill the original object with values from the surrogate.

The circular serializer does more then just solve the circular serialization problem. I might refactor this to make it clearer.

Editable Business Object (only on client):

  1. this project implements IEditableObject (and INotifyPropertyChanged) on your domain object
  2. knows how to keep 'original values' and provide them to the serializer

Again, this project might be split.


So there are 3 different parts here. Let's look at that with a picture:


Basically, the circular serializer is the glue that is used on both the server and the client. It knows how to serialize your objects and asks for help to get to the original values. On the client, the EditableBusinessObject can provide that, on the server, it is the original POCO project.

Using them is dead simple, as I'll show.

Look at some code

I've used the School database and let EF generate the entity mappings. I did not change anything from the defaults, which basically means that the classes we end up with are pretty awful. I don't care about them though for this sample.


I then set the generated code to 'not compile' and created the following beautiful classes by hand:

  0 namespace Domain
  1 {
  2 [DataContract(Name = "Person", Namespace = "tstNS")]
  3  public class Person
  4 {
  5   [DataMember]
  6   public int PersonID { get; set; }
  7   [DataMember]
  8   public string LastName { get; set; }
  9   [DataMember]
  10   public string FirstName { get; set; }
  11   [DataMember]
  12   public DateTime? HireDate { get; set; }
  13   [DataMember]
  14   public DateTime? EnrollmentDate { get; set; }
  16   [DataMember]
  17   public ICollection<Enrollment> Enrollments { get; set; }
  18   [DataMember]
  19   public ICollection<Course> Courses { get; set; }
  21 }
  23 [DataContract(Name = "Course", Namespace = "tstNS")]
  24  public class Course
  25 {
  26   [DataMember]
  27   public int CourseID { get; set; }
  28   [DataMember]
  29   public string Title { get; set; }
  30   [DataMember]
  31   public string Days { get; set; }
  32   [DataMember]
  33   public DateTime Time { get; set; }
  34   [DataMember]
  35   public string Location { get; set; }
  36   [DataMember]
  37   public int Credits { get; set; }
  38   [DataMember]
  39   public Department Department { get; set; }
  41   [DataMember]
  42   public ICollection<Enrollment> Enrollments { get; set; }
  43   [DataMember]
  44   public ICollection<Person> Persons { get; set; }
  46 }
  47 [DataContract(Name="Department", Namespace="tstNS")]
  48  public class Department
  49 {
  50   [DataMember]
  51   public int DepartmentID { get; set; }
  52   [DataMember]
  53   public string Name { get; set; }
  54   [DataMember]
  55   public Decimal Budget { get; set; }
  56   [DataMember]
  57   public DateTime StartDate { get; set; }
  58   [DataMember]
  59   public int Administrator { get; set; }
  61   [DataMember]
  62   public ICollection<Course> Courses { get; set; }
  64 }
  65 [DataContract(Name = "Enrollment", Namespace = "tstNS")]
  66  public class Enrollment
  67 {
  68   [DataMember]
  69   public int EnrollmentID { get; set; }
  70   [DataMember]
  71   public Decimal? Grade { get; set; }
  73   [DataMember]
  74   public Course Course { get; set; }
  75   [DataMember]
  76   public Person Person { get; set; }
  78 }
  79 }

Now, I need a trick to get seperate versions of this on the client and the server. I altered the msbuild script to add a conditional compile symbol that is the name of the solution. You could go fancy with this technique, by matching the build name and stuff. I just used this:


(Please let me know when someone uses this technique to target both Silverlight and regular .net. It's possible!)
So, now it's time to apply our postsharp attributes. I did that in the assembly info file:

#if Server
[assembly: Poco("SchoolEntitiesConnection", AttributeTargetAssemblies = "Domain", AttributeTargetTypes = "Domain.*")]
[assembly: CreateSerializeSurrogate("EntityFrameworkContrib.PostSharp4EF.ISerializationHelp, EntityFrameworkContrib.PostSharp4EF", AttributeTargetAssemblies = "Domain", AttributeTargetTypes = "Domain.*")]
#if Client
[assembly: EditableBusinessObjectAttribute(AttributeTargetAssemblies="Domain", AttributeTargetTypes="Domain.*")]
[assembly: CreateSerializeSurrogate("EditableBusinessObjects.Postsharp.Public.IEditableBusinessObject, EditableBusinessObjects.Postsharp.Public", AttributeTargetAssemblies="Domain", AttributeTargetTypes="Domain.*")]

Pretty neat, isn't it? This notifies postsharp to either use Poco or EditableBusinessObject to modify the compiled code. I only have one place to maintain this code.
As you can see, the circular serialize (CreateSerializeSurrogate) takes the name of a type that it can use to get 'original value' information from.

I also created an interface project (just a nice best-practice) to identify my operation contract:

  0 [CircularReferenceSurrogateAttribute]
  1 [ServiceContract(Namespace="Http://sitechno.School")]
  2  public interface ISchoolService
  3 {
  4   [OperationContract]
  5   Person GetPerson(int id, bool fetchEnrollments, bool fetchCourses);
  7   [OperationContract]
  8   void SavePerson(Person person);
  10   [OperationContract]
  11   IList<Course> GetCourses();
  13   [OperationContract]
  14   IList<Department> GetDepartments();
  16 }

See the CircularReferenceSurrogateAttribute on line 0: it applies a datacontract surrogate manager, that will substitute surrogate types for the real types during serialization.

Now, for the server:

  0  public class SchoolService : ISchoolService
  1 {
  2   public Person GetPerson(int id, bool fetchEnrollments, bool fetchCourses)
  3   {
  4    using (SchoolContext context = new SchoolContext())
  5    {
  6     Person p = (from person in context.Person
  7        where person.PersonID  == id
  8           select person).First();
  10     if (fetchEnrollments)
  11     {
  12      ((IRelationshipLoader)p).Load("Enrollments");
  13      foreach (Enrollment r in p.Enrollments)
  14      {
  15       ((IRelationshipLoader)r).Load("Course");
  16       ((IRelationshipLoader)r).Load("Person");
  17      }
  18     }
  20     if (fetchCourses)
  21     {
  22      ((IRelationshipLoader)p).Load("Courses");
  23      foreach (Course c in p.Courses)
  24      {
  25       ((IRelationshipLoader)c).Load("Department");
  26       ((IRelationshipLoader)c).Load("Enrollments");
  27       ((IRelationshipLoader)c).Load("Persons");
  29      }
  30     }
  32     ((IContextAware)p).CreateSerializableState(context);
  34     return p;
  35    }
  37   }
  40   public void SavePerson(Person person)
  41   {
  42    using (SchoolContext context = new SchoolContext())
  43    {
  44     ((IContextAware)person).AttachGraphToContext(context, entity => entity.GetType().Name);
  46     context.SaveChanges();
  47    }
  48   }
  51   public IList<Course> GetCourses()
  52   {
  53    using (SchoolContext context = new SchoolContext())
  54    {
  55     List<Course> courses = context.Course.ToList();
  56     courses.ForEach(crs => ((IContextAware)crs).CreateSerializableState(context));
  57     return courses;
  59    }
  60   }
  62   public IList<Department> GetDepartments()
  63   {
  64    using (SchoolContext context = new SchoolContext())
  65    {
  66     List<Department> deps = context.Department.ToList();
  67     deps.ForEach(dp => ((IContextAware)dp).CreateSerializableState(context));
  68     return deps;
  69    }
  70   }
  72 }

Before someone tells me that I'm not doing the includes/spans correctly in the GetPerson method: I'll do it better when I do a more complete sample.

As you can see, for methods that return classes, I call the CreateSerializableState method. Methods that accept a graph have to call the AttachGraphToContext method. Obviously, these can and should be context extensibility methods.

  • The CreateSerializableState method will use the context to get the 'original' values and load them into your object. Your class actually has fields to hold this information (as created by Poco).
  • The AttachGraphToContext method is quite complex and will look at a graph and it's original values. It will first set the original values, than do an acceptchanges and change only those properties that were actually changed. It also fixes up relationships, using add/attach/delete.

The client:

  0   static void Main(string[] args)
  1   {
  2    SchoolService svc = new SchoolService();
  4    IList<Department> departments = svc.GetDepartments();
  6    Person Fadi = svc.GetPerson(4, false, false);
  8    Console.WriteLine(String.Format("Working on {0} {1}, Hired at {2:D}",
  9     Fadi.FirstName, Fadi.LastName, Fadi.HireDate));
  10    Console.WriteLine();
  12    Console.WriteLine("The ID is not generated by the database in this instance, so please give me a number that is not used already");
  13    int id = Int32.Parse(Console.ReadLine());
  15    Console.WriteLine("I can bug you for a title as well, can't I?");
  16    string title = Console.ReadLine();
  18    Course newCourse = new Course { CourseID = id, Credits = 10, Days = "MT", Time = DateTime.Now, Title = title };
  19    newCourse.Persons = new List<Person> { Fadi }; // normally, your object ctor would have done this
  20    newCourse.Enrollments = new List<Enrollment>();
  21    newCourse.Department = departments.First();
  22    newCourse.Department.Courses.Add(newCourse);
  24    Fadi.FirstName = "Fadi,Changed at " + DateTime.Now.ToShortTimeString();
  27    Fadi.Courses.Add(newCourse);
  30    svc.SavePerson(Fadi);
  32    Console.WriteLine("Finished, go check your database. I added the new course to department:" + newCourse.Department.Name);
  33    Console.ReadLine();
  36   }

One important thing to know is that on the client, there is no longer a concept of relations, only lists. So you will need to hook up classes from both angles when you connect them: see lines 21 and 22.

Things to note
  • There is a LOT going on, but you are shielded from it and can work in a blissful persistence ignorance way!
  • Your objects will implement quite a few interfaces that you can use if you'd like
  • The client side implements INotifyPropertyChanged (and raises events correctly) but the ICollection implementations are still just Lists<>. I will change that to ObservableCollections for you WPF lovers! (which includes me)
  • I have not implemented deleting yet. So you can unhook a relationship, but that does not delete the object. I don't think that it should either! I will probably implement some 'MarkAsDelete' method on the client-side
  • There is a strong optimization for speed. I, for instance, generate a method 'SetValue(string propname, object val)' and corresponding GetValue so that the framework does not need to use reflection or dynamic method generation. Still, I haven't covered everything yet, so there is still some expensive reflection going on. The overall goal is certainly to reduce this to null.
  • This is not a version 1.0 release. Do not treat it as such. It's complex matter, and some things are not supported yet. Still 98% is finished.
  • I rushed the samples. There is a wpf sample that has the worst codebehind that exists. Do not open it, your eyes will hurt. When I have time I'll build a proper MVC sample that actually doesn't crash.

I am very excited to have gotten this far. Since PostSharp supports Silverlight, I'll try to get this to work for silverlight as well, so we can build some serious RIA's! I'll try to follow up with a screen cast soon.

kick it on

Monday, 07 April 2008 21:25:31 (Romance Standard Time, UTC+01:00)  #    Comments [5]  |  Trackback
 Friday, 21 March 2008

I'm working hard on bringing a couple of my projects together that will enable domainobjects on the client without Entity Framework references. I thought I'd give a small update on how that's working out for me.

The endresult

What I want is one visual studio project: 'Domain' that holds business objects for me that I can use on both the client and the server. What I do not want, is to have to think about EntityFramework contexts on the client. Since I do not persist my objects on the client, I see no reason to actually create a context on the client, just to be able to do changetracking. Nor do I want to see any EF specific plumbing in the messages between that client and server: I lose interoperability if I do that. Also, if my client is Silverlight (for instance) I might not want to bring the EF assemblies over (if that's even possible?).

So, the endresult should be an easy way to serialize and deserialize objects on the client and server, and a way to build up the object correctly on the server so it can be attached to the context again.

Please check Daniel's EntityBag sample for a very sophisticated way of serializing your context. He does go down the route of serializing the objectcontext, which does mean you need the entityframework on the client. Read about his motivations for doing this here.

Let's see it

I'll first show some code and then walk you through it.

  0    using (OneSimpleType.OneSimpleTypeConnection context = new OneSimpleType.OneSimpleTypeConnection())
  1    {
  2     // clear out database
  3     foreach (OneSimpleType.Person old in context.Person)
  4     {
  5      context.DeleteObject(old);
  6     }
  7     context.SaveChanges();
  8    }
  10    OneSimpleType.Person p = new OneSimpleType.Person { Firstname = "Ruurd", Lastname = "Boeke" };
  11    IEditableBusinessObject ep = p as IEditableBusinessObject;
  12    string MsgOnWire = "";
  14    using (OneSimpleType.OneSimpleTypeConnection context = new OneSimpleType.OneSimpleTypeConnection())
  15    {
  16     context.AddToPerson(p);
  17     context.SaveChanges();  // at this point, there should be 'original values'
  19     // the server is changing the object without yet saving
  20     p.Firstname = "ServerChanged";
  22     // now we need to create a version of the object that can be serialized well
  23     p = (OneSimpleType.Person)context.CreateSerializableVersion(p);
  25     DataContractSerializer s = new DataContractSerializer(p.GetType(), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  27     MsgOnWire = GetWellFormedToContract(p, s);
  29    }
  31    // imagine we are the client getting a message and deserializing it
  32    DataContractSerializer s2 = new DataContractSerializer(typeof(OneSimpleType.Person), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  33    StringReader reader = new StringReader(MsgOnWire);
  34    XmlReader xreader = XmlReader.Create(reader);
  35    OneSimpleType.Person p2 = (OneSimpleType.Person)s2.ReadObject(xreader);
  37    // edit something on the client
  38    p2.Lastname = "ClientChanged";
  41    // we have edited everything, let's get back to the server
  42    s2 = new DataContractSerializer(typeof(OneSimpleType.Person), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  43    MsgOnWire = GetWellFormedToContract(p2, s2);
  45    // we're at the server
  46    using (OneSimpleType.OneSimpleTypeConnection context = new OneSimpleType.OneSimpleTypeConnection())
  47    {
  48     DataContractSerializer s3 = new DataContractSerializer(typeof(OneSimpleType.Person), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  49     StringReader reader3 = new StringReader(MsgOnWire);
  50     XmlReader xreader3 = XmlReader.Create(reader3);
  51     OneSimpleType.Person p3 = (OneSimpleType.Person)s3.ReadObject(xreader3);
  53     context.AttachedDeserializedVersion(p3, "Person");
  55     context.SaveChanges();
  56    }

We start out on the serverside.
On line 17, we have just saved an object to the database. 
We continue using that (attached, and thus changetracked) object and change a property (just for the fun of it).
On line 27 we serialize it like you've seen me do in previous posts.

On line 32, we pretend to be on the client, which uses the message to deserialize to a compatible type. Note, that this type does not need to be 'IPoco' enhanced. It does need to be enhanced for serialization and editing.

We change a property 'on the client' on line 38. Notice that this object does not have a context attached at all.
At line 43 we have once again serialized this new object and on line 46 we pretend to be on the server again.

There we create a new context and deserialize our message to an object. Then we call an extension method of context to attach this version to the context. Note that I have to pass the setname (working on somehow getting to this information differently).
Finally we save.

Only the changed properties are flagged as dirty, so efficient SQL can be generated by EF.

The extension method is shown here:

  0   public static void AttachedDeserializedVersion(this ObjectContext context, object source, string setname)
  1   {
  2    // TODO: how to get setname from metadata or something
  4    Debug.Assert(source is IEntityWithKey);
  5    Debug.Assert(source is IEditableBusinessObject);
  7    IEditableBusinessObject eS = (IEditableBusinessObject)source;
  9    // we go to the loaded state
  10    eS.SetReadMode(ReadMode.Loaded);
  14    string fullEntitySetName = context.DefaultContainerName + "." + setname;
  16    EntityKey createdKey = context.GetEntityKey(fullEntitySetName, source);
  17    ((IEntityWithKey)source).EntityKey = createdKey;
  19    // attach this version
  20    context.Attach((IEntityWithKey)source);
  22    eS.ReplayLoadedToNormal();
  24    eS.SetReadMode(ReadMode.Normal);
  27   }

You see I have quite a bit of functionality in the IEditableBusinessObject interface.

The deserialized version of the object has a 'loaded' state (or better: original state) and a 'current' state. Before attaching at line 20, I first set my object to be in 'original' mode. Then I have a replay method that will compare the current values to the loaded versions and touch the setters of the properties of the changed versions. This notifies the changetracker of EF to flag a property as dirty.

This all works fine, but both code and api are pretty crude. I have no complextype or relationshiptype support, but that will come, if I don't run into major problems along the way.

I hope you are as excited about this as I am. Let me know!

Friday, 21 March 2008 18:21:28 (Romance Standard Time, UTC+01:00)  #    Comments [2]  |  Trackback
 Monday, 17 March 2008

I'll probably dedicate a bigger post to this soon, but I wanted to show you a domainmodel, some code and the xml it generates.
What you see here is two attributes that take on quite a bit of work.
EditableBusinessObject implements IEditableObject for you and also allows you to copy the currentvalues to a loadedvalues state.
CreateSerializeSurrogate generates a surrogate class that knows how to deal with loadedvalues and with circular references.
Together, they form the backbone of the client-side of your domainmodel.

Here is an example.

(notice this is the full class, no other properties are here)

  0 [EditableBusinessObject]
  1 [CreateSerializeSurrogate]
  2 [Serializable]
  3 [DataContract(Namespace = "myNamespace", Name = "Person")]
  4  public class Person
  5 {
  6   [DataMember]
  7   public int IntProperty { get; set; }
  9   [DataMember]
  10   public string StringProperty { get; set; }
  12   [DataMember]
  13   public string StringProperty2 { get; set; }
  15   int neverSetInt;
  16   [DataMember]
  17   public int NeverSetInt
  18   {
  19    get
  20    {
  21     return neverSetInt;
  22    }
  23    set { neverSetInt = value; }
  24   }
  26   [DataMember]
  27   public List<string> StringLijst { get; set; }
  29   [DataMember]
  30   public List<int> IntLijst { get; set; }
  32   public Person()
  33   {
  34    StringLijst = new List<string>();
  35    IntLijst = new List<int>();
  36   }
  38 }

Some testcode:
(notice line 16 where we do a endEdit. If we had cancelled, we would've had a proper rollback)

  0    Person p = new Person();
  1    p.IntProperty = 1;
  2    p.StringProperty = "Ruurd";
  3    p.StringProperty2 = "Boeke";
  5    p.StringLijst.Add("a");
  6    p.IntLijst.Add(1);
  8    (p as IEditableBusinessObject).CopyCurrentToLoaded();
  10    (p as IEditableBusinessObject).BeginEdit();
  12    p.IntLijst.Add(2);
  13    p.StringLijst.Add("b");
  14    p.StringProperty = "Ruurd Boeke";
  16    (p as IEditableBusinessObject).EndEdit();
  18    DataContractSerializer s = new DataContractSerializer(p.GetType(), null, int.MaxValue, false, false, new SubstituteDomainDataContractSurrogate());
  20    string outMessage = GetWellFormedToContract(p, s);

And the generated xml:
(notice how the lists and 'StringProperty' are changed, and see the original value).

<PersonSurrogate xmlns:i="" xmlns="myNamespace">
    <IntLijst xmlns:d2p1="">
    <StringLijst xmlns:d2p1="">
    <StringProperty>Ruurd Boeke</StringProperty>
    <OriginalValue_IntLijst xmlns:d2p1="">
    <OriginalValue_StringLijst xmlns:d2p1="">

What is left, is deserializing, maybe propagating the beginedit commands to all children and then create the serverside EF variant.
Oh, and I don't like the originalValue representation. Maybe I'll generate a OriginalValue class to keep it tidy.

Monday, 17 March 2008 22:28:31 (Romance Standard Time, UTC+01:00)  #    Comments [11]  |  Trackback
 Tuesday, 11 March 2008

In a previous post, I presented a PostSharp weaver that created a surrogate class for all your domain classes to help in serializing circular references (you know: Father.Children and Child.Father will cause WCF to go into a loop). The important enabler there was a List<object> that I held onto while (de)serializing. I was unhappy with the way I had solved that, so in this post, I'll just fix that.

Part of the problem was that the Server (dispatcher) has a correctly initialized OperationContext, where I could get to OperationContext.Current and do stuff. Extensions that were added in an earlier phase were no longer accessible though. A shame really.
Nicholas Allen was able to tell me how I could get a OperationContext going in the client. You can just create an OperationContext. I created a proxy class (by hand) that does this. People who love generated code are out of luck on this one, but I would recommend against using the generation tools of VS for WCF anyway.

My Proxyclass:

  0  public class PersonSvcClient : ClientBase<IPersonService>, IPersonService
  1 {
  2   public PersonSvcClient() : base ("PersonEndpoint")
  3   {
  5   }
  6   #region IPersonService Members
  8   public void SendPersonGraphToService(Person person)
  9   {
  10    using (new OperationContextScope(this.InnerChannel))
  11    {
  12     base.Channel.SendPersonGraphToService(person);
  13    }
  14   }
  16   public Person GetPersonToClient()
  17   {
  18    using (new OperationContextScope(this.InnerChannel))
  19    {
  20     return base.Channel.GetPersonToClient();
  21    }
  22   }
  25   #endregion
  26 }

As you can see, I'm putting the actual calls inside a using block that just creates an operationcontextscope.

Now, if I were to subscribe to the OperationContext.Current.OperationCompleted, you would think that I had a great place to clean up. However, this event does not get thrown on the client (it works correctly on the server..). Weird.

However, now I have an OperationContext.Current in both client as server, so I can add an extension to it. Beware, I add this extension right in the surrogate. This way, it will stay around during the serialization process.

  0   public class MyContext : IExtension<OperationContext>
  1   {
  2    public static MyContext GetCurrentContext()
  3    {
  4     MyContext c = OperationContext.Current.Extensions.Find<MyContext>();
  5     if (c == null)
  6     {
  7      OperationContext.Current.Extensions.Add(c = new MyContext());
  8     }
  10     return c;
  11    }
  13    /// <summary>
  14    /// will keep our objects during serialization. Need to clear this before serialization (operation behavior)
  15    /// </summary>
  16    public List<object> serializationList = new List<object>();
  18    /// <summary>
  19    /// will keep our objects during desrialization. Need to clear this before deserialization (operation behavior)
  20    /// </summary>
  21    /// <remarks>the key is the serialization ID that is transmitted by the </remarks>
  22    public Dictionary<int, object> deserializationList = new Dictionary<int, object>();
  24    #region IExtension<OperationContext> Members
  26    public void Attach(OperationContext owner)
  27    {
  28    }
  30    public void Detach(OperationContext owner)
  31    {
  32    }
  34    #endregion
  35   }

I've added a static GetCurrentContext method, that will add the context class if it does not exist, and return it.

Finally, now I can be sure that I have a safe place to store my lists, and it get's garbage collected properly.

Tuesday, 11 March 2008 14:19:12 (Romance Standard Time, UTC+01:00)  #    Comments [12]  |  Trackback
 Monday, 03 March 2008

Ever since I was using domain objects on the wire and using WCF httpbindings to serialize them, I've had one serious issue with the way cyclic / circular references are serialized. In this post I present a solution that will create a more beautiful XML representation of your object graph than ever before!
As you know, I'm a big proponent of easy Domain Driven Design and flexibility. So, the solution I present you will be supereasy to implement (just use an attribute).

The issue at hand

Let's create two objects that have references to each other:

    [DataContract(Namespace = "myNamespace", Name = "Person")]
    public class Person
        public int PersonID { get; set; }
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public int Something;
        public List<int> Numbers { get; set; }
        public List<int> FieldedNumbers;
        public List<Order> Orders { get; set; }
        public Person()
            Orders = new List<Order>();
            Numbers = new List<int>();
            FieldedNumbers = new List<int>();
    [DataContract(Namespace = "myNamespace", Name = "Order")]
    public class Order
        public Person Customer { get; set; }
        public int Amount { get; set; }
        public int ProductID { get; set; }
        public int OrderID { get; set; }

The diagram thus looks like this:


When you use a regular DataContractSerializer to serialize this, you will get a stack overflow, since there is a circular reference between person and order.

  1         [TestMethod]
  2         public void TestWithoutSurrogateSubstitution()
  3         {
  4             Person p = new Person { PersonID = 23, FirstName = "Ruurd", LastName = "Boeke", Something = 666 };
  5             p.Numbers.Add(31);
  6             p.Numbers.Add(29);
  8             p.FieldedNumbers.Add(12);
  9             p.FieldedNumbers.Add(24);
 11             p.Orders.Add(new Order { Amount = 12, Customer = p, OrderID = 12, ProductID = 19 });
 12             p.Orders.Add(new Order { Amount = 12, Customer = p, OrderID = 13, ProductID = 255 });
 14             DataContractSerializer s = new DataContractSerializer(p.GetType(), null, int.MaxValue, false, true, null);
 16             string outMessage = GetWellFormedToContract(p, s);
17         }

So, on line 14, you will see that I have to instantiate my serializer with the 'preserveObjectReferences' boolean set to true. The outputmessage of line 16 is shown here:

<Person xmlns:i="" z:Id="1" xmlns:z="" xmlns="myNamespace">
    <FieldedNumbers xmlns:d2p1="" z:Id="2" z:Size="2">
    <FirstName z:Id="3">Ruurd</FirstName>
    <LastName z:Id="4">Boeke</LastName>
    <Numbers xmlns:d2p1="" z:Id="5" z:Size="2">
    <Orders z:Id="6" z:Size="2">
        <Order z:Id="7">
            <Customer z:Ref="1" i:nil="true" />
        <Order z:Id="8">
            <Customer z:Ref="1" i:nil="true" />

Now, I have had long conversations with the PM of WCF, in the days it was still called Indigo, about the way this is serialized. I think it was harder to actually set the boolean back then, because the xml created here is not platform independent. Look at all the z:ref and z:ID attributes. I hate them!!

This serialization method is only understood by WCF, so you need both WCF on the client and on the server.
That's not really an issue for most, and there does not exist a standard for serializing object graphs. So there just is not a way to solve this issue. However, there is a way to be more friendly and get rid of the ugly z:ID and z:Ref attributes!!

Interested?? Read on.

My Solution: create surrogates on the fly

I use PostSharp to create a surrogate type during the compilation phase. Then I use a DataContractSurrogate to substitute these during serialization and deserialization.

All you need to do is attach that surrogate on your service (I have created an attribute for that) and use an attribute on your domainclasses to inform PostSharp to do it's magic:

    [DataContract(Namespace = "myNamespace", Name = "Person")]
    public class Person
        public int PersonID { get; set; }

So, the new XML that is being sent on the wire, looks like this:

  1 <Person xmlns:i="" xmlns="myNamespace">
  2     <FieldedNumbers xmlns:d2p1="">
  3         <d2p1:int>12</d2p1:int>
  4         <d2p1:int>24</d2p1:int>
  5     </FieldedNumbers>
  6     <FirstName>Ruurd</FirstName>
  7     <LastName>Boeke</LastName>
  8     <Numbers xmlns:d2p1="">
  9         <d2p1:int>31</d2p1:int>
 10         <d2p1:int>29</d2p1:int>
 11     </Numbers>
 12     <Orders>
 13         <Order>
 14             <Amount>12</Amount>
 15             <Customer>
 16                 <SerializationID>0</SerializationID>
 17             </Customer>
 18             <OrderID>12</OrderID>
 19             <ProductID>19</ProductID>
 20             <SerializationID>1</SerializationID>
 21         </Order>
 22         <Order>
 23             <Amount>12</Amount>
 24             <Customer>
 25                 <SerializationID>0</SerializationID>
 26             </Customer>
 27             <OrderID>13</OrderID>
 28             <ProductID>255</ProductID>
 29             <SerializationID>2</SerializationID>
 30         </Order>
 31     </Orders>
 32     <PersonID>23</PersonID>
 33     <SerializationID>0</SerializationID>
 34     <Something>666</Something>

35 </Person>

That's much better!

Look at line 33, where our person is assigned an ID. Now, checkout lines 16 and 25: Our Order object is pointing to our already serialized Person class.

How does it work
The CreateSerializeSurrogate attribute

If you have read my posts about PostSharp, you know that it is a framework that is able to alter the assemblies you create after they have been compiled. The CreateSerializeSurrogate instructs PostSharp to create a nested type inside your domainobject, called xxxSurrogate. That surrogate class has a constructor taking in the original type. It copies all values to it's own values. However, it is also passed a List<object>. It checks whether the original object was already serialized. If it was, no values will be copied, except the serializationID. So, at line 15 for instance, you see a complete person instance serialized, but just without it's values.

PostSharp also introduces a Deserialize(List<object>) method, that will do the reverse.

Let's take a look at the result using reflector:

  1 [DataContract(Namespace="myNamespace", Name="Person")]
  2 public class Person
  3 {
  4     // Fields
  5     [CompilerGenerated]
  6     private string <FirstName>k__BackingField;
  7     [CompilerGenerated]
  8     private string <LastName>k__BackingField;
  9     [CompilerGenerated]
 10     private List<int> <Numbers>k__BackingField;
 11     [CompilerGenerated]
 12     private List<Order> <Orders>k__BackingField;
 13     [CompilerGenerated]
 14     private int <PersonID>k__BackingField;
 15     [DataMember]
 16     public List<int> FieldedNumbers;
 17     [DataMember]
 18     public int Something;
 20     // Methods
 21     static Person();
 22     public Person();
 23     public void CopyDataFromSurrogate(PersonSurrogate surrogate);
 25     // Properties
 26     [DataMember]
 27     public string FirstName { [CompilerGenerated] get; [CompilerGenerated] set; }
 28     [DataMember]
 29     public string LastName { [CompilerGenerated] get; [CompilerGenerated] set; }
 30     [DataMember]
 31     public List<int> Numbers { [CompilerGenerated] get; [CompilerGenerated] set; }
 32     [DataMember]
 33     public List<Order> Orders { [CompilerGenerated] get; [CompilerGenerated] set; }
 34     [DataMember]
 35     public int PersonID { [CompilerGenerated] get; [CompilerGenerated] set; }
 37     // Nested Types
 38     [DataContract(Name="Person", Namespace="myNamespace")]
 39     public class PersonSurrogate
 40     {
 41         // Fields
 42         [DataMember(EmitDefaultValue=false)]
 43         public List<int> FieldedNumbers;
 44         [DataMember(EmitDefaultValue=false)]
 45         public string FirstName;
 46         [DataMember(EmitDefaultValue=false)]
 47         public string LastName;
 48         [DataMember(EmitDefaultValue=false)]
 49         public List<int> Numbers;
 50         [DataMember(EmitDefaultValue=false)]
 51         public List<Order> Orders;
 52         [DataMember(EmitDefaultValue=false)]
 53         public int PersonID;
 54         [DataMember]
 55         public int SerializationID;
 56         [DataMember(EmitDefaultValue=false)]
 57         public int Something;
 59         // Methods
 60         public PersonSurrogate();
 61         public PersonSurrogate(Person source, List<object> graphList);
 62     }
 63 }

I have chosen not to expand methods. See how there is a complete PersonSurrogate class, that you will not find in your sourcecode. Also, a CopyDataFromSurrogate exists! The constructor on line 61 is shown here:

public PersonSurrogate(Person source, List<object> graphList)
    if (!graphList.Contains(source))
        this.PersonID = source.PersonID;
        this.FirstName = source.FirstName;
        this.LastName = source.LastName;
        this.Numbers = source.Numbers;
        this.Orders = source.Orders;
        this.Something = source.Something;
        this.FieldedNumbers = source.FieldedNumbers;
    int ListID = graphList.IndexOf(source);
    this.SerializationID = ListID;

Well, there you go.

Doing this in PostSharp was not the easiest thing, but also not the hardest. I did it, so you don't have to. Let's look at a small piece of the weaver:

        private void CopyFieldsToSurrogate(InstructionWriter writer)
            foreach (FieldDefDeclaration field in source.Fields)
                if (field.CustomAttributes.FirstOrDefault(attr => attr.ConstructRuntimeObject() is DataMemberAttribute) != null)
                    // has datamember
                    FieldInfo finfo = field.GetReflectionWrapper(null, null);
                    FieldDefDeclaration f = new FieldDefDeclaration();
                    f.Name = field.Name;
                    f.FieldType = field.FieldType;
                    f.Attributes = System.Reflection.FieldAttributes.Public;
                    // copies the datamember attribute in the sourcecollection
                    CopyDataMemberAttribute(field.CustomAttributes, f.CustomAttributes);
                    // init the field
                    writer.EmitInstructionField(OpCodeNumber.Ldfld, field);
                    writer.EmitInstructionField(OpCodeNumber.Stfld, f);

Ouch. I did not use the highlevel Laos, but used the core functionality to really insert IL. This means there is no runtime overhead at all.

The datasetsurrogate

The tricky part was to get a list to the objects during serialization. I have created a datacontractsurrogate for that, and an attribute to place it on your service:  [AttachSurrogateAttribute]

I'm not particularly pleased with the internals of the surrogate. I ran into some big problems getting a context for the operation: it is discarded right before the datacontractsurrogate goes to work, and the client-side does not have a operationcontext.current at all. So I had trouble getting a solution that will maintain a list only for the time I need it.

I had to power-through it. However there is absolutely some extension point I have overlooked that would work for this.

For now, I have two surrogates: one for the client (which does not use operationcontext.current) and one for the service. It works great though. I'll figure out a better way, or comment me if you know one.


If you are happy with the representation of the xml that is sent by the preserveObjectReferences boolean, then that's fine. If you are not, the downloadable solution below will fix this quite handily. If you need to interop with some other platform, I think they can come to grips with your contract.
I just want to send over nice xml. ;-)


You can download the solution here. No need to install PostSharp or anything.
Remember, the Server project is going to self-host, so if you want to start it, start VS in admin mode.
Now, if you want to see it work, you can either start both Server and Client projects and press the nice buttons. Set breakpoints to see that your graph went over just fine.
Use the unittests to see how the graph looks.

[disclaimer: I made this project in a rush. As I said the datacontractsurrogate was not implemented nicely. Also, I know I could have done better by implementing an interface on the created surrogate class, so I could call it's methods better..  If there is interest in these things, I might spend more time on it. For now, I'll leave it at this]

[disclaimer 2: forgot to mention: this will probably not work on generic domain objects. If there is interest, I will spend the time to make that work]


kick it on

Monday, 03 March 2008 14:32:14 (Romance Standard Time, UTC+01:00)  #    Comments [6]  |  Trackback
 Friday, 08 September 2006

I originally posted this little titbit in the wcf forums, but since lots of people there did not seem to know about it, I'll post it here again:
The framework 3.0 RC1 install will kill your existing .svc mapping, so you will not be able to communicate with IIS hosted WCF services (.svc). Obvious workaround is to add the mapping again manually. This will be fixed in the next RC.

I hope that helps a few people out there!

Friday, 08 September 2006 21:25:26 (Romance Standard Time, UTC+01:00)  #    Comments [0]  |  Trackback
 Wednesday, 07 June 2006

Ouch, this really got to me. I upgraded to the May CTP, hacked my application to compile again and ran it.

The first screen of the application let's you choose a person to load. That went fine. But when loading more of this persons data on a following screen, Indigo completely freezes. That puzzled me quite a bit. Everything worked fine with the Feb. CTP, and I had not expected serialization problems. Wcf has a way of not throwing exceptions with serialization problems so I did not really have much to go on. I was retrieving a collection with about 14 objects in it. When I deleted a few objects everything went smooth again! So something new with maximum message size was going on?

The solution was not to up the maximum size (it was high enough), but to loosen the reader Quota's for my service.
A reader Quota is defined as: 'Gets or sets constraints on the complexity of SOAP messages that can be processed by endpoints configured with this binding'. The constraints include items such as a maximum element depth, a maximum length for string content within the message. The constraints protect against a class of denial of service (DOS) attacks that attempt to use message complexity to tie up endpoint processing resources.

There you have it. I guess they lowered the defaults.
<readerQuotas maxDepth="90000" maxStringContentLength="90000"
            maxArrayLength="90000" maxBytesPerRead="90000"
            maxNameTableCharCount="90000" />

That helped me for some time, but recently I've had to go higher again. Possibly the Wcf team could set the defaults to much higher values!

Wednesday, 07 June 2006 12:35:16 (Romance Standard Time, UTC+01:00)  #    Comments [0]  |  Trackback