Sitelock...

I have a set of solutions for an application that were to be upgraded to a new release. I use a setup feature that activates/re-activates necessary features, runs som jobs, modifies web.config etc. All this to minimize failures caused by manual implementation. The hosting company just re-installs the solutions, activates the activate setup and everything is up'n running in a minute. Perfect.

But this upgrade failed. When activated using stsadm, we didn't get any errors but my logfiles constantly reported AccessDenied exceptions. When activated using the Central Administration UI, a ThreadAborted exception was thrown.

After looking into my loggfiles I noticed that everthing didn't actually fail. Some features were actually activated and timerjobs started successfully. A little more digging showed me that every read operation in the site collection worked fine, but the write operations failed with an AccessDenied exception.

Permission errors? Not very likely, I tried to run the process using all possible accounts including the farm account. The ULS log didn't provide much help. Only a line with the AccessDenied exception.

Ok, turn on verbose logging... The ULS said something like "PermissionMask check failed. asking for 0x00000005 ... "

What the...?

But it gave me the answer:

http://social.technet.microsoft.com/Forums/en/sharepointadmin/thread/742debc0-8bef-4f89-8c4a-77d291fd39e5

The site collection was locked, presumably after a server failure. Unlocked the site and everything ran smoothly!

More about the upgraded Content Query Web Part

A lot of the previous posts here are related to issues I found when I upgraded a farm from MOSS to SP2010. One of the most common web part used in the main site collection is the Content Query Web Part (CQWB). Unfortunately we've had plenty of strange issues with this web part. Some things that worked fine before the upgrade failed.

Some web parts didn't return any items (All these web parts are predefined in xml and added to sites via onet or manually as separate web parts). I think I've mentioned before that CQWBs scoped to a specific list should use the internal name of the fields specified in most places, but for CQWBs scoped to cross-list queries should use the id of the fields. In MOSS the sorting failed if the sort field was specified with an internal name so that field was specified with the field id, but fields in queries, common viewfields, column renames etc were specified with the field names. After upgrading most of the failing web parts could be fixed by using the correct field specification type, i.e. we exported the web parts, adjusted the xml and replaced the failing ones.

But more issues came up after a while. Some web parts crashed the page when we should edit it via UI. After a lot of struggle I found the failure. In the web part xml, four fields were specified as common view fields. The ULS said that the xml-definition for some fields couldn't be found. Those fields had nothing to do with this, but got me in the correct path. The actual exception that crashed the page was not caused by the web part, but it's toolpart. When the textboxes that displays the common view fields (not available in MOSS) are populated, the field identifiers are retrieved from the web parts xml-definition and fetched as SPField objects to get the display name.

In the MOSS days things worked fine even if the specified common view fields actually existed or not (to be honest I haven't actually looked at the code, but that was the behaviour), but now the fields MUST exist in the correct scope, i.e. in the list if the query is set to a specific list. Using dotPeek I read through loads of sharepoint (2010) code and found that they in one critical place didn't check for a possible null exception.
When the toolpart shall populate the textboxes that displays the DisplayName of the DataMappings or CommonViewFields, it uses the populateFieldsToDisplayControlGroup to loop through the fields specified to get the DisplayName. I haven't looked too deep inside the DataMapppings part of the loop, but when it comes to the CommonViewFields it calls the getDataMappingField method where str1 is the specified field name to get the SPField object and passes it to the ContentByQueryWebPart.GetFieldTitle method to get the DisplayName.

In the getFieldFromCollection method we can se that the first try to get the SPField object is by the Id and the second try is by the internal name. But the method can also return a null if the field doesn't exist in the SPFieldCollection passed from the getDataMappingField method (i.e. SPList.Fields in the case of a single list scope). And so the ContentByQueryWebPart.GetFieldTitle can get a null as an input parameter and it throws a null reference exception when it returns the field.Title value.

    public class ContentByQueryWebPart : CmsDataFormWebPart, INamingContainer, IWebPartVariationUpdate
    { 
... 
        internal static string GetFieldTitle(SPField field)
        {
            if (field.Id == ContentByQueryWebPartConstants.catchAllId)
                return Resources.GetDocumentManagementString("KeyFiltersKeywordsCatchAll_AllTags");
            else
                return field.Title;
        }
 
        internal static SPField getFieldFromCollection(Guid fieldId, string internalName, SPFieldCollection fields)
        {
            SPField spField = (SPField) null;
            try
            {
                if (fieldId.CompareTo(Guid.Empty) != 0)
                    spField = fields[fieldId];
            }
            catch (ArgumentException ex)
            {
            }
            try
            {
                if (spField == null)
                {
                    if (!string.IsNullOrEmpty(internalName))
                        spField = fields.GetFieldByInternalName(internalName);
                }
            }
            catch (ArgumentException ex)
            {
            }
            return spField;
        }

...

    }


    public sealed class ContentByQueryToolPart : ToolPart
    {
...
        private void populateFieldsToDisplayControlGroup(bool clearInvalidMappings)
        {
            ArrayList allSlotNames = this.contentByQueryWebPart.AllSlotNames;
            Hashtable stringAsHashTable1 = ContentByQueryToolPart.getStringAsHashTable(this.contentByQueryWebPart.DataMappings, '|', ':');
            Hashtable stringAsHashTable2 = ContentByQueryToolPart.getStringAsHashTable(this.contentByQueryWebPart.CommonViewFields, ';', ',');
            int index1 = 0;
            while (index1 < allSlotNames.Count)
            {
                string str1 = allSlotNames[index1] as string;
                string str2 = string.Empty;
                if (stringAsHashTable1.Contains((object) str1))
                {
                    string input = stringAsHashTable1[(object) str1] as string;
                    if (clearInvalidMappings && "LinkUrl".Equals(str1))
                    {
                        StringBuilder dataMappingResult = new StringBuilder();
                        string[] strArray1 = input.Trim().Split(new char[1] {';'});
                        int index2 = 0;
                        while (index2 < strArray1.Length)
                        {
                            string str3 = strArray1[index2].Trim();
                            if (!string.IsNullOrEmpty(str3))
                            {
                                string[] strArray2 = str3.Trim().Split(new char[1] {','});
                                Guid fieldId = Guid.Empty;
                                string internalName = string.Empty;
                                if (strArray2.Length > 1)
                                {
                                    fieldId = new Guid(strArray2[0]);
                                    internalName = strArray2[1];
                                }
                                SPField dataMappingField = this.getDataMappingField(fieldId, internalName, SPContext.Current.Web);
                                if (this.isSafeDataMapping(dataMappingField, str1))
                                    ContentByQueryToolPart.addFieldIntoDataMapping(dataMappingField, dataMappingResult, (StringBuilder) null);
                            }
                            checked
                            {
                                ++index2;
                            }
                        }
                        input = ((object) dataMappingResult).ToString();
                    }
                    str2 = this.getDisplayedDataViewMapping(SPContext.Current.Web, input);
                }
                else if (stringAsHashTable2.ContainsKey((object) str1))
                    str2 = ContentByQueryWebPart.GetFieldTitle(this.getDataMappingField(Guid.Empty, str1, SPContext.Current.Web));
                this.fieldsToDisplayTextBoxes[index1].Text = str2;
                checked
                {
                    ++index1;
                }
            }
        }

        private SPField getDataMappingField(Guid fieldId, string internalName, SPWeb web)
        {
            return this.contentByQueryWebPart.CurrentQueryScope() != ContentByQueryWebPart.QueryScope.List ? ContentByQueryWebPart.getFieldFromCollection(fieldId, internalName, web.AvailableFields) : ContentByQueryWebPart.getFieldFromCollection(Guid.Empty, internalName, this.targetList.Fields);
        }
...
    }

My fields does exist as site columns but they doesn't exist in the list, so the solution is to either add the fields to the list or to remove them from the CommonViewFields in the web part.

Deleting a SPPersistedObject

A really annnoying behaviour concerning persisted objects: even though the object fails to serialize when you update it the first time to save it (maybe you've forgot to create a default constructor or you use types that are not serializable), it will still be saved even though you get an error. The catastrophy happens when you try to get it back ( e.g. SPFarm.Local.GetObject(Id) ) to delete it. You get an error saying that the object cannot be deserialized and thus you can't delete the object.

Another strange this is that when I try to add or get another object that serializes / deserializes correctly, sharepoint, it seems, tries to deserialize the failing object aswell and so I can't use any objects.

I tried the "The undocumented deleteconfigurationobject parameter" but that gave the same error. I had to use "The (totally unsupported) solution" mentioned here and delete it manually from the database :(

Different CQWP rendering behavior when upgrading to SP2010

In the past I started to love and hate the Content Query Web Part since you can use it with audience targeting. In the MOSS days you had a very good control of what was happening and what to render. A rather annoying behavior though was that - even in rather normal situations - the query didn't give you the result you wanted.

My customer wanted to render an audience filtered calendar that showed the upcoming 7 events from [Now] and beyond, not [Today] and beyond. Since there is no such thing as <Now/> that you can use in a caml query, I used <Today/>, increased the row limit and in the item style (ItemStyle.xsl) filtered out the items not matching the rules. But sometimes I still didn't get the 7 events I wanted. I found out that the audiene filtering is not included in the query, but is made on the result of the query. So if I requested 20 items, the audience filtering maybe took away 15 items and the rendering rules in the xsl maybe took away 2. So instead of the expected 7 items I got 3.

The only simple solution to solve this was to increase the row limit even more and it worked fine.

But now the behaviour of the rendering is quite different. In theold days the result was rendered as a <table> but now it's a <ul><li> thing. Better HTML, but it has issues. When you render a table row with only empty cells it doesn't show anything. but an empty <li> in a <ul> renders an empty row. So when I filter out an item in the item style it still renders an empty row since the <li> tag is rendered in ContentQueryMain.xsl.
My solution to this was to set the style attribute in the <li> to display:none;. And how to do that? First the <li> tag must be rendered as a tag, not as a text string that is the case in the default ContentQueryMain.xsl (the $BeginListItem variable). In the OuterTemplate.CallItemTemplate I rendered an <li class="dfwp-item"> tag instead of <xsl:value-of disable-output-escaping="yes" select="$BeginListItem" /> so that the last rendered tag is a fully qualified xml-tag. In my item style, if the item doesn't match the rules, I set the attribute of thea fully qualified xml-tag. In my item style, if the item doesn't match the rules, I set the style attribute of the last rendered xml-tag (i.e. the <li>-tag) with <xsl:attribute name="style">display:none;</xsl:attribute>.

Works beautifully!

Audience targeting failing after upgrade to SharePoint 2010 pt2

In the former post I described an issue that can arise when upgrading MOSS to SP2010 concerning audience targeting. The solution in most cases turns out to be to change the name attribute in fields from Target_x0020_Audiences to Audience. In som cases that will not work. For uncustomized list schemas containing items, the new field named Audience is actually just added to the list instance.

That will of course result in major issues since there are two fields in the list with the same id but with different names. When opening an item in such a list, the aspx-form will crash. My only solution to this was to make  create a temporary list based on the same template (where the audience field has the correct name), copy all items from the original list, delete the original list and create a new one with the same name, copy the items back and delete the temporary list.

The issue here being that all references to the original list will be lost, like List View Web Parts or CQWPs pointed to that specific list.

I went for this solution because time became critical and the customer accepted to make the manual patches.

Audience targeting failing after upgrade to SharePoint 2010

This is not the final solution! In some cases it can be, but in my case it turned out to fail. More coming up...

If you're upgrading to SP2010 and used audience targeting in MOSS for lists simply by enabling it in the UI or by using code, you can stop reading now. But if you enabled it by adding the Target_x0020_Audiences field in a custom list schema, this can be of interest!

I'm upgrading a customer's intranet from MOSS to SP2010. A great deal of information is displayed using CQWP together with Target Audiences.

In a earlier project for the same customer I moved this intranet from one farm to another and then I began to realize how SharePoint actually handles audience targeting. Since audiences are handled by the SSP in MOSS, the definitions and IDs are stored in the SSP database (in a table called something like OrgleList), and since the audience set in a list item is stored with the ID, the connection breaks when moving the content database for the web application to another farm where there is another SSP dealing with the audiences. Even if the definitions are the same, they get a new ID and so everything fails. I've published some code to handle this at codeplex: Audience Migration .

But now I'm upgrading to SP2010 and the upgrade handles this almost perfectly (or at least as perfect as in can get) since I also upgrade the SSP database when setting up the User Profile Service. "almost" means ofcourse that there's been a horrible change in the Publishing features that enable the use of audience targeting:

Sometimes my Content Query Web Parts (CQWPs) in the upgraded intranet showed nothing when I specified that audience filtering should be applied, and it showed all items when I also included untargeted items even though they were targeted. Not the same behaviour as in the old MOSS intranet. So I tried to create new CQWPs to make my head a little bit clearer. And yes, I found out that if I set the query to point to a specific list, the correct behaviour sometimes fails, but if I set it to use the site collection or a web it worked fine... Strange... Ok, start ILSpy:

When the CQWP is building the query to execute, it loops through properties like CommonViewFields, DataMappingViewFields and SystemViewFields to aggregate the ViewFields-tag in the query. The actual query is registered in the ULS and I noticed that sometimes the query used the field id to specifiy a FieldRef in the ViewFields-tag and sometimes it uses the internal name. And yes, when the query scope is set to a specific list it uses internal names and otherwise it uses the field IDs. When looking in the code it should get the field name from the SPList.Fields collection, but the query looks for the field called Audience even though the field used for audience targeting in the list is named Target_x0020_Audiences. But I also noticed that before looking for the field in the list, the code tries to get the field name from a thread-safe cache...

But how come that the query looks for a field named Audience when it should be Target_x0020_Audiences?
In MOSS the Target Audiences field, defined in the PublishingResources feature, was named Target_x0020_Audiences so i looked in the file to make sure. And yes, they changed it!!! Now it's simply named Audience. The ID is fortunately the same.

Now we're getting somewhere. Some of the list schemas in the intranet are customized (unghosted), i.e. the schema xml is saved in the database. When mounting the MOSS content database, the upgrading process corrects the field names in the customized lists, but since the schema of an uncustomized list is outside of the database, the upgrade process can't correct this.

The effect is that if you have several CQWPs in a page and some points to specific lists and at least one of those lists is customized or using the new field name Audience, the behaviour of the web parts will depend on which is rendered first. If an upgraded customized list (or a new list with "correct" field name Audience) is rendered first, the field name Audience will be cached and used as view field in that query and every following query of other CQWPs in that page! If an old uncustomized list is rendered first, the others with correct field names will fail when it comes to audience filtering.

So, after a long time of looking around, the solution was simple: don't forget to change the Name and StaticName of the Target_x0020_Audiences field to Audience. I don't think many will ever notice this problem. When you manually enable audience targeting in a list, the list schema will be customized and saved in the database since the Audience field is added to the list. In my case, I enabled audience targeting in my lists by including the field in the schema.xml file.

Hope I can help someone out there!
Cheers!

User Profile Service troubles

Most of you reading this probably have a lot of broken harware after smashing screens, keyboards and god knows what. Setting up the User Profile stuff in SharePoint 2010 is hell. But after tips and tricks you actually might succeed! Look at this to get details of how it really works http://www.harbar.net/articles/sp2010ups.aspx
Followed it by the numbers but still, things got stuck when starting the User Profile Synchronization Service. ULS said a lot of strange stuff, like a service was too busy or that the proxy for User Profile Service application couldn't be found.
A few broken keyboards later I found a strange thing in the event viewer for security. An audit failure for the user profile service account. A classic NULL SID, and the DisableLoobackCheck trick in the registry did not solve it. A closer look at the failure reason: "The user has not been granted the requested logon type at this machine.". The logon type here is 4... googling... Aha, found it! A lot of posts out there says that the farm account must be able to logon locally and that it is set via the local security policy. But logon type 4 does NOT mean "logon locally". It means "log on as a batch job"!
Looking in the local security policy shows that the Performance Log Users group is allowed to logon as a batch job. A little reminder in my head rang a bell. I recall seeing that somewhere out there (wouldn't surprise me if it is the microsoft documentation :) ) Another reminder told me something about the default schema set in the databases (Profile DB, Social DB, Sync DB). Some people out there say something about a bug and that the default schema should be set to dbo.
Trying all this... I can now feel safe to buy a nice keyboard!

That stuff about the Performance Log Users group seems to be important for other service applications aswell. Have no idea why these things aren't set correctly in the current farm though.

Cheers!