Audience targeting failing after upgrade to SharePoint 2010

This is not the final solution! In some cases it can be, but in my case it turned out to fail. More coming up...

If you're upgrading to SP2010 and used audience targeting in MOSS for lists simply by enabling it in the UI or by using code, you can stop reading now. But if you enabled it by adding the Target_x0020_Audiences field in a custom list schema, this can be of interest!

I'm upgrading a customer's intranet from MOSS to SP2010. A great deal of information is displayed using CQWP together with Target Audiences.

In a earlier project for the same customer I moved this intranet from one farm to another and then I began to realize how SharePoint actually handles audience targeting. Since audiences are handled by the SSP in MOSS, the definitions and IDs are stored in the SSP database (in a table called something like OrgleList), and since the audience set in a list item is stored with the ID, the connection breaks when moving the content database for the web application to another farm where there is another SSP dealing with the audiences. Even if the definitions are the same, they get a new ID and so everything fails. I've published some code to handle this at codeplex: Audience Migration .

But now I'm upgrading to SP2010 and the upgrade handles this almost perfectly (or at least as perfect as in can get) since I also upgrade the SSP database when setting up the User Profile Service. "almost" means ofcourse that there's been a horrible change in the Publishing features that enable the use of audience targeting:

Sometimes my Content Query Web Parts (CQWPs) in the upgraded intranet showed nothing when I specified that audience filtering should be applied, and it showed all items when I also included untargeted items even though they were targeted. Not the same behaviour as in the old MOSS intranet. So I tried to create new CQWPs to make my head a little bit clearer. And yes, I found out that if I set the query to point to a specific list, the correct behaviour sometimes fails, but if I set it to use the site collection or a web it worked fine... Strange... Ok, start ILSpy:

When the CQWP is building the query to execute, it loops through properties like CommonViewFields, DataMappingViewFields and SystemViewFields to aggregate the ViewFields-tag in the query. The actual query is registered in the ULS and I noticed that sometimes the query used the field id to specifiy a FieldRef in the ViewFields-tag and sometimes it uses the internal name. And yes, when the query scope is set to a specific list it uses internal names and otherwise it uses the field IDs. When looking in the code it should get the field name from the SPList.Fields collection, but the query looks for the field called Audience even though the field used for audience targeting in the list is named Target_x0020_Audiences. But I also noticed that before looking for the field in the list, the code tries to get the field name from a thread-safe cache...

But how come that the query looks for a field named Audience when it should be Target_x0020_Audiences?
In MOSS the Target Audiences field, defined in the PublishingResources feature, was named Target_x0020_Audiences so i looked in the file to make sure. And yes, they changed it!!! Now it's simply named Audience. The ID is fortunately the same.

Now we're getting somewhere. Some of the list schemas in the intranet are customized (unghosted), i.e. the schema xml is saved in the database. When mounting the MOSS content database, the upgrading process corrects the field names in the customized lists, but since the schema of an uncustomized list is outside of the database, the upgrade process can't correct this.

The effect is that if you have several CQWPs in a page and some points to specific lists and at least one of those lists is customized or using the new field name Audience, the behaviour of the web parts will depend on which is rendered first. If an upgraded customized list (or a new list with "correct" field name Audience) is rendered first, the field name Audience will be cached and used as view field in that query and every following query of other CQWPs in that page! If an old uncustomized list is rendered first, the others with correct field names will fail when it comes to audience filtering.

So, after a long time of looking around, the solution was simple: don't forget to change the Name and StaticName of the Target_x0020_Audiences field to Audience. I don't think many will ever notice this problem. When you manually enable audience targeting in a list, the list schema will be customized and saved in the database since the Audience field is added to the list. In my case, I enabled audience targeting in my lists by including the field in the schema.xml file.

Hope I can help someone out there!
Cheers!

User Profile Service troubles

Most of you reading this probably have a lot of broken harware after smashing screens, keyboards and god knows what. Setting up the User Profile stuff in SharePoint 2010 is hell. But after tips and tricks you actually might succeed! Look at this to get details of how it really works http://www.harbar.net/articles/sp2010ups.aspx
Followed it by the numbers but still, things got stuck when starting the User Profile Synchronization Service. ULS said a lot of strange stuff, like a service was too busy or that the proxy for User Profile Service application couldn't be found.
A few broken keyboards later I found a strange thing in the event viewer for security. An audit failure for the user profile service account. A classic NULL SID, and the DisableLoobackCheck trick in the registry did not solve it. A closer look at the failure reason: "The user has not been granted the requested logon type at this machine.". The logon type here is 4... googling... Aha, found it! A lot of posts out there says that the farm account must be able to logon locally and that it is set via the local security policy. But logon type 4 does NOT mean "logon locally". It means "log on as a batch job"!
Looking in the local security policy shows that the Performance Log Users group is allowed to logon as a batch job. A little reminder in my head rang a bell. I recall seeing that somewhere out there (wouldn't surprise me if it is the microsoft documentation :) ) Another reminder told me something about the default schema set in the databases (Profile DB, Social DB, Sync DB). Some people out there say something about a bug and that the default schema should be set to dbo.
Trying all this... I can now feel safe to buy a nice keyboard!

That stuff about the Performance Log Users group seems to be important for other service applications aswell. Have no idea why these things aren't set correctly in the current farm though.

Cheers!

Load control template file /_controltemplates/TaxonomyPicker.ascx failed

Around the world people say that the solution is to replace , with a comma in the Inherits property in the Control tag. Tried it but it doesn't resolve the issue. Also some people are saying that the class actually doesn't exist anymore (at least not in any that namespace or assembly). Yes, that is the correct "solution". I just wright this to make sure that there are as many hits as possible pointing to the correct answer.

Disposing issues...

So many annoying things concerning disposing SPRequest based objects like SPWeb and SPSite. You can assume that objects like these should be disposed by SharePoint when they are given to you in properies in other objects, like SPListItem.Web, but no...

Examples I've found:

When you do a call to get an SPListItem from an SPItemEventProperties (typically in a receiver like properties.ListItem), if it's the first time you call for the list item, the get method opens up a new SPWeb object to get the list item based in the ListId and ListItemId properties and returns the list item. But the SPWeb object is never disposed. You have to manually dispose it from the list item.

When you call SPListItem.Attachments a similar situation occurs. the SPWeb object in the SPListItem.Web propery gets re-initialized and then you have to dispose it manually.

Pretty annoying ey?

The ListItemProperty Web Control

The issue: in the forms.aspx page, the <SharePoint:ListItemProperty> should render the Title field in the SPListItem. Sure it works. Almost as you would expect. My title is 255 characters but the MaxLength property is set to 40 in the forms.aspx. Nice. But no... All characters are rendered. Why? Search on google: nothing. Reflector (thanks Lutz Roeder for saving my life so many times!): yes!

Looking again at the web control you might notice that the Property property is not set to "Title" so you can assume that it is the default property to render, and yes it is. But when looking at the code you can see that if the property isn't set to anything, the control renders simply SPContext.Current.ListItemDisplayName. It doesn't truncate the output to match the MaxLength property. But when the Property property is set in the web control, the Render method uses the SPWeb.Request.RenderColumn method which takes all properties, including MaxLength.

So since it actually affects all lists you could actually edit the actual form.aspx file in the Pages folder in the hive. I don't since it's a out-of-the-box file, it's simple enough to use a copy.

Authorized Images in an FBA Login Page

My first post. I intend to use this blog mostly just to remind myself about useful stuff I've done, but if it can help anybody else I'll be happy.
A customer wants to a have a login page but still using authentication from their AD. First I thought, not possible since NTLM just allows the normal windows login popup. A former colleague said that I maybe could consider making it a forms based authentication application using the ActiveDirectoryMembershipProvider together with SSL. It was ok for the customer but maybe that will be another post.
The Big problem was to design the login page. With their current oracle-based solution it would take up to two weeks to get a change in the login page. Requests to the hosting company, decisions, meetings etc, and ok they accept to change the . to a , .
Now a user can edit the the login page using an announcement item in a list. The login page runs a code with elevated privileges to get the latest announcement and places the Title and Body fields in the page in the Render method. Nice.
But after a while they noticed that they couldn't use an image in the body. Hmmm... why not? Aha, the image is a link to an attachment in the item. Damn! The login page gets the correct html, but since the user isn't logged in yet, the img-tags can't load the images.
First I thought a solution would be to take a "snap shot" of the body including the images and put it in the LAYOUTS folder and set the location as allowed in web.config. But when I tried to write the stuff to LAYOUTS I noticed that not even the Application Pool account have write access.
Ok, another idea. Replace the url to the image with a url to an aspx-file that takes the image's url in the query string, runs code with elevated privileges to get the image and in the Page.Response writes the image as a byte[].
I still had to set that the aspx-file is allowed non-authenticated access in web.config. But hey, it worked! But just to be careful I made the url in the query string as a Base64Encoded string (of course I could have made a somewhat more safe encoding, but time is money).

So, the solution was:
An aspx-file namned getimage.aspx placed in the LAYOUTS folder containing

<%@ Assembly Name="MyProject, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3334a3f90ce5bdd0" %>
<%@ Page Language="C#" Inherits="MyProject.Code.ApplicationPages.GetImage" %>
 
The class MyProject.Code.ApplicationPages.GetImage
 
public class GetImage : Microsoft.SharePoint.WebControls.UnsecuredLayoutsPageBase

{
protected override void OnLoad(EventArgs e)
{
  if (string.IsNullOrEmpty(Request.QueryString["url"]))
    return;

  SPSecurity.RunWithElevatedPrivileges(() =>
  {
    string siteUrl = string.Format("{0}://{1}", HttpContext.Current.Request.Url.Scheme, HttpContext.Current.Request.Url.Host);
    string url = Base64Decode(Request.QueryString["url"]);
    using (SPSite site = new SPSite(siteUrl))
    {
      using (SPWeb web = site.OpenWeb())
      {
        SPFile file = web.GetFile(url);
        Page.Response.BinaryWrite(file.OpenBinary(SPOpenBinaryOptions.None));
      }
    }
  });
}
private static string Base64Decode(string sBase64String)
{
  byte[] sBase64String_bytes = Convert.FromBase64String(sBase64String);
  return UnicodeEncoding.UTF8.GetString(sBase64String_bytes);
}

And the really tricky part: correcting the urls in the body in the login page. It's a little part in the Render method (The class is called MyProject.Code.ApplicationPages.LoginPage and inherits from Microsoft.SharePoint.ApplicationPages.LoginPage):

protected override void Render(System.Web.UI.HtmlTextWriter writer)
{
  SPSecurity.RunWithElevatedPrivileges(() =>
  {
    string url = string.Format("{0}://{1}", HttpContext.Current.Request.Url.Scheme, HttpContext.Current.Request.Url.Host);
    using (SPSite site = new SPSite(url))
    {
      using (SPWeb web = site.RootWeb)
      {
        var list = web.GetList("/lists/loginmessages");
        var query = new SPQuery();
        query.Query = "<Where><Geq><FieldRef Name=\"Expires\"/><Value Type=\"DateTime\"><Today/></Value></Geq></Where><OrderBy><FieldRef Name=\"Expires\" Ascending=\"FALSE\"/></OrderBy>";
        var items = list.GetItems(query);
        if (items.Count > 0)
        {
          lblTitle.Text = items[0].Title;
          string body = Convert.ToString(items[0][SPBuiltInFieldId.Body]);
          Regex regex = new System.Text.RegularExpressions.Regex("(?<=\\<img.*src=\")((/lists/loginmessages/attachments/\\d*/)[-a-zA-Z0-9@:%_\\+.~#?&//=]+)\\.(jpg|jpeg|gif|png|bmp|tiff|tga|svg)", RegexOptions.IgnoreCase);
          body = regex.Replace(body, "/_layouts/getimage.aspx?url=$&");
          regex = new Regex("(?<=\\<img.*src=\"/_layouts/getimage\\.aspx\\?url=)((/lists/loginmessages/attachments/\\d*/)[-a-zA-Z0-9@:%_\\+.~#?&//=]+)\\.(jpg|jpeg|gif|png|bmp|tiff|tga|svg)", RegexOptions.IgnoreCase);
          body = regex.Replace(body, new MatchEvaluator(LoginPage.ReplaceEncode));
          litBody.Text = body;
        }
      }
    }
  });
}

and the methods to do the encoding in the login page:

private static string ReplaceEncode(Match match)
{
  return match.Result(Base64Encode(match.Value));
}
private static string Base64Encode(string sString)
{
  byte[] sString_bytes = System.Text.UnicodeEncoding.UTF8.GetBytes(sString);
  return Convert.ToBase64String(sString_bytes);
}