SharePoint governance

Introduction

I do a lot of SharePoint implementations in both big and small companies. The one and only problem I almost always see coming back is the governance of the platform. How do we partition our system, which sites do we create but most importantly, how do we let the system grow and how do we engage people in using SharePoint and make SharePoint the central platform for Enterprise Content Management within the organization.

When I’m working for clients, I always talk to a varied group of people and what they think the current function of SharePoint is in their company. This was one of the reasons for me to write this post. It really bothers me that a lot of organizations, for some very easy identifiable reasons, are not using the full potential of this platform.

Please … SharePoint is not just a portal

I see many many (did I say many) clients for which a portal is one of the biggest functions of the SharePoint platform. On this portal they spread all the company news for the employees to read. Although a portal on SharePoint is very useful and definitely something you would want to do on this platform, it sometimes bothers me that the decision makers fail to see the bigger picture.

Management … you must step in

The biggest reason why SharePoint is often not fully used, is because on the level of enterprise content management, the management is not always taking the right decisions or is not taking decisions at all. For me, ideally, SharePoint would be the one and only platform for users to work with business related digital data. This means that SharePoint is the only platform they use to consult, create and edit data. It is our role as consultants to make all of this work and to extend the platform in such a way that all systems are integrated in the best way possible.

What I often see in companies is the complete opposite. People have too many choices of managing their digital company data. They can use e-mail, other ECM platforms, shares, … . You can’t blame the people. Why would they change their old habit of using network shares ? They have done this for many years and still have the opportunity to continue doing it.

Honestly, if your company has a SharePoint platform, how many times per day you still receive an email with a document attached ?

Therefore the following rules should be applied:
- The company must make sure that a fully integrated SharePoint platform is available for all employees to work with their digital company data.
- The company must make sure that no other means of storing or sharing documents can happen. This means: no network shares, no other ECM platforms and no emailing of documents to each other. You always upload the document to SharePoint, provide proper metadata and use the link to the document as a means of distributing it.
If we take this rule even one step further: people should not even have documents stored on their personal hard drives anymore.
- The company must provide (internal) SharePoint courses to make people familiar with basic SharePoint principles and how to work with it.

How to let the system grow

There are many aspects to this topic. You can see it from many different perspectives and they all need their specific guidance. For me there are three parts in this:

1. The architecture

- Make sure your initial SharePoint farm is powerful enough and has enough capacity to handle the number of employees in the company. Also make sure the grow margin on the hard disk level is big enough to support the growth of the system in the near future.
- Hire professional SharePoint administrators which will do the initial architecture and installation of the platform. Find them internally or externally and let them train other people in parallel. It is imperative your initial SharePoint installation happens by the books. It’s nice you have big plans for SharePoint within your company but if the architecture is not right, how can you ever build something solid on top of it ? Therefore, hire the right people to do the job.

2. Consultants

I admit that consultant is a rather broad term but what I mean here is a very competent technical person (programmer) with good soft skills. Being a consultant means that people pay you to give them advice. Therefore, make sure you know what you are talking about so you can give them good advice. For me, there are some very basic rules you must obey when talking about governance:

- Create your solutions in a uniform way. By this I mean: agree on coding conventions and make sure the solutions (GUI), looks the same. If SharePoint is going to be the one and only ECM platform, it must be presented as one solid platform. Almost like a product.
- Make your solutions user friendly. Somewhat related to the ‘use the same GUI’ story but going farther than that. Because technical people often do not care about user friendliness, it might be a good idea to get more artistic people involved.
People are having already enough problems with the big and radical switch to SharePoint and loosing their network shares and email. The last thing they want is a platform that looks and behaves the same in different locations (sites) and having to use a user unfriendly tool.
- Training: one aspect of this is pure SharePoint training were consultants explain people the basics of SharePoint and its functions. Listen to the people. Don’t start by summing up everything they might be able to do with SharePoint. Different people in the company have different tasks and needs. This is also a good opportunity to do business development. People will most likely ask for extra extensions of the platform to make their lives easier. Explain them the advantages of using SharePoint. You have metadata, search, workflows, audit, … .
The other aspect of this is explaining, giving training, about the custom extensions of the SharePoint platform. Some development teams just develop what was requested (of what they think was requested) and then just let the customer know: “Ok, here it is, have fun”. This is really really bad. What you should do is go back to your customer, sit together with the business owner and the main users and show/explain them the solution.

3. Power users

This is a hard one. Especially in big companies, SharePoint platforms have this intension of becoming … well .. spaghetti. Sites are being created everywhere, let’s also add some lists and content types, workflows, … .

Although business owners would like to have this kind of permissions (managing their own site to the fullest) and although this might also be the easiest from an administration/programmer point of view, this is also the first step in ending up in this spaghetti state. A lot of these power users just don’t know enough of SharePoint to make the right decisions. By this I don’t necessarily mean they don’t know enough about what SharePoint can do for a power user, I mean they don’t have the top view of the complete SharePoint farm. They don’t know which content types already exist, which workflows have already been developed before, … .

The solution is very simple: limit the permissions of the power users. In my opinion, only a handful of people in the entire organization should have the permission to add new content types, create and manage workflows, … . This would mean that every request for such an extension would go through some kind of process where it’s assessed if the solution has already been developed somewhere else and if the extension itself is necessary in the first place.

I have seen many many cases where power users start doing stuff (I would better say trying out stuff) and half way they are stuck. Then they need help. They ask you to take a look and provide input. You assess the problem, you ask the right questions to figure out what they exactly try to accomplish and provide a clean, simple, sometimes out of the box solution.

My point is that if too many people have to many (=high) permissions, it is almost guaranteed your SharePoint platform will become hard to manage. Limiting the permissions and applying correct processes for the implementation of extensions, will increase control, will increase quality, stability, user acceptance and will allow you to have a long term vision with SharePoint as your central ECM platform.

Nintex Workflow 2010 bug: 'Value does not fall within expected range'

Installed version: Nintex Workflow 2010 (2.2.3.0)

I was recently installing a solution on the client's farm. Part of the solution was a collection of Nintex workflows. When I imported a workflow and tried to publish it, I received the following message (this also happened with very simple '1 action workflows' that were newly created).

After doing tests for several hours we concluded it had to do with the number of items in the library the workflow was imported in. The default threshold for lists is 5000 items. We had like 8200 items, so more than the configured threshold. When increasing the threshold above 8200, the importing and publishing went just fine.

Of course this has some implications and therefore I cannot really agree with how the Nintex code is written (did some reflector digging). According to Nintex this has to do with SharePoint limitations and I really feel this is not a correct answer. First of all, the 5000 limit threshold has to do with how many items a view in a list can return. It says nothing about how many items you can have in the list itself. Even after creating views and making sure not one view returned more than the 5000 limit threshold, the problem would persist.

It was on this moment I decided to take a look into their code to find out what is going on. The stack trace was pointing to the following functions:

   1: private static SPFolder GetMetadataFolder(SPList list)

   2: {

   3:     SPFolder result = null;

   4:     if (list.BaseType == SPBaseType.DocumentLibrary)

   5:     {

   6:                     result = list.RootFolder.SubFolders["Forms"];

   7:     }

   8:     else

   9:     {

  10:                     result = list.RootFolder;

  11:     }

  12:     return result;

  13: }

  14:  

  15: public static AutoStartRuleCollection LoadFromList(SPList list)

  16: {

  17:     SPFolder metadataFolder = AutoStartRuleCollection.GetMetadataFolder(list);

  18:     return AutoStartRuleCollection.ReadRulesFile(metadataFolder, "Nintex_AutoStartRules.xml");

  19: }

  20:  

  21: private static AutoStartRuleCollection ReadRulesFile(SPFolder folder, string filename)

  22: {

  23:     SPFile file = folder.ParentWeb.GetFile(SPUrlUtility.CombineUrl(folder.Url, filename));

  24:     if (!file.Exists)

  25:     {

  26:                     return null;

  27:     }

  28:     byte[] bytes = file.OpenBinary();

  29:     string serialised = Utility.ConvertByteArrayToString(bytes);

  30:     return AutoStartRuleCollection.Deserialize(serialised);

  31: }

The exception itself is thrown by line 6 of the first function. Because the ‘SubFolders’ property is used, in the background the threshold limit is kicking in. What they actually try to do here is construct an URL to a file called ‘Nintex AutoStartRules.xml’ and for doing this, they use the SPFolder class. The file is stored in the hidden ‘Forms’ folder in document libraries and in the root for custom lists.

I have tried other ways to get a hold of this ‘Forms’ folder without triggering the threshold limit but came to the conclusion it’s not possible (as far as I know). But when looking at the bigger picture (they just want the construct the URL to that file), we could conclude that the URL can be easily constructed without the need of the SPFolder class which would make the solution work in all circumstances because the xml file always has the same name and the location is known upfront based on the type of the list.

Creating a WCF service in SharePoint 2010 using JSON

Recently I was trying to create a REST WCF service which uses JSON and although there are a lot of articles that talk about it, I couldn’t find one which contained a totally working example for SharePoint 2010. So to spare you some hours of frustration, I will share this information with you .

1. Download the CKS tools
Because Visual Studio 2010 doesn’t have out of the box templates to quickly develop a WCF service, I downloaded and installed the CKS tools.

2. Create WCF Service
In your project, add a new WCF service from the CKS templates. You will notice that the template will always create an interface and a SVC + code behind file.

3. Add your method prototypes to the interface
Open the interface file and add prototypes for each of your service methods.

   1: namespace MyNamespace

   2: {

   3:     // NOTE: You can use the "Rename" command on the "Refactor" menu to change the interface name "ISubscriptionService" in both code and config file together.

   4:     [ServiceContract]

   5:     public interface IWCFService1

   6:     {

   7:  

   8:         [OperationContract]

   9:         [WebInvoke(Method = "POST", RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Wrapped)]

  10:         MyMethodAnswer MyMethod(string p1, int p2);

  11:  

  12:     }

  13:  

  14: }

What’s the most important here is the WebInvoke tag. It specifies the method uses POST, that the JSON format is used to accept and return data and that the sent data must be wrapped in a wrapping element (more about that later).

4. Edit the SVC file

By default it points to the code behind file. Let’s change it to the full assembly name. I also got rid of the Factory property.

   1: <%@ ServiceHost Language="C#" Debug="true" 

   2: Service="MyNamespace.MyClass, MyAssemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d373e6657f243d25" %>

5. Implement your methods.

As you can see, the method is returning a ‘MyMethodAnswer’ object. You can use this if you want to return more than just a primitive type. Just make sure that object can be serialized. These methods are implemented in the ‘MyClass’ class.

   1: public MyMethodAnswer MyMethod(string p1, string p2)

   2: {

   3:     

   4:     // Implement

   5:  

   6: }

6. Prepare the web.config

I always create a separate web.config which holds the WCF service’s configuration. It’s better to deploy the SVC file in it’s separate subfolder in the ISAPI folder (automatically done by Visual Studio). The important thing is to also deploy the web.config there. Therefore I also create an ISAPI/MyAssemblyName folder in the Visual Studio project. In there, I create the web.config.

   1: <?xml version="1.0"?>

   2: <configuration>

   3:  

   4:   <system.serviceModel>

   5:  

   6:     <!-- we need this to enable session -->

   7:     <serviceHostingEnvironment aspNetCompatibilityEnabled="true" />

   8:     <behaviors>

   9:       <endpointBehaviors>

  10:  

  11:         <!-- our configuration for rest relies on web http -->

  12:         <behavior name="RestBehavior">

  13:           <webHttp />

  14:         </behavior>

  15:  

  16:       </endpointBehaviors>

  17:  

  18:       <serviceBehaviors>

  19:         <behavior name="NewBehavior" >

  20:           <serviceMetadata httpGetEnabled="true" />

  21:           <serviceDebug

  22:             httpHelpPageEnabled="true"

  23:             includeExceptionDetailInFaults="true"

  24:           />

  25:         </behavior>

  26:       </serviceBehaviors>

  27:  

  28:     </behaviors>

  29:     <bindings>

  30:       <webHttpBinding>

  31:  

  32:         <!-- a selection of security bindings that you can use in the service registration below-->

  33:         <binding name="WindowsAuthenticationBasicHttpBinding">

  34:           <security mode="TransportCredentialOnly">

  35:             <transport clientCredentialType="Windows" />

  36:           </security>

  37:         </binding>

  38:         <binding name="NoSecurityHttpBinding">

  39:           <security mode="None">

  40:             <transport clientCredentialType="None" />

  41:           </security>

  42:         </binding>

  43:  

  44:       </webHttpBinding>

  45:  

  46:     </bindings>

  47:     <services>

  48:  

  49:       <service name="MyNamespace.MyClass" behaviorConfiguration="NewBehavior">

  50:         <endpoint address=""

  51:                 binding="webHttpBinding"

  52:                 behaviorConfiguration="RestBehavior"

  53:                 contract="MyNamespace.IMyClass"

  54:                 bindingConfiguration="WindowsAuthenticationBasicHttpBinding">

  55:         </endpoint>

  56:       </service>

  57:  

  58:     </services>

  59:  

  60:   </system.serviceModel>

  61:  

  62: </configuration>

First of the all the web.config contains information about how the end point and the web service behave. This is defined in the behavior tags. Please note that the behaviors are used in the service definition.

In the bindings we define two authentication methods. The first one is the standard Windows authentication. The second one has no authentication.

Last thing is the service itself. The name itself doesn’t really matter. Just make sure you correctly reference the behavior configuration in the service and endpoint tag.

Make sure the ‘contract’ property is correctly set. This must point to your interface class in the correct namespace.

7. Deploy

What’s left right now is packaging and deploying it. You can then surf to the SVC file using your browser. If everything is right, you will see a page that starts with

   1: You have created a service.

   2:  

   3: To test this service, you will need to create a client and use it to call the service. You can do this using the svcutil.exe tool from the command line with the following syntax:

   4:  

and what follows is some info about how to talk with your service.

8. Test the communication

Building the service is one thing, communicating with it is something else. It’s not always clear what exact data the service expects and what kind of data it will send back. Here are some tips to help you:

- In the same folder as the SVC file, create an html file with some simple JQuery:

   1: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

   2: <html>

   3:     <head>

   4:         <title></title>

   5:         <script type="text/javascript" src="http://code.jquery.com/jquery-latest.js"></script>
   1:  
   2:     </head>
   3:     <body>
   4:     
   5:         <script>
   6:  
   7:             jQuery.support.cors = true;
   8:  
   9:             var url = 'http://myserver/_vti_bin/mySubFolder/MyService.svc/MyMethod';
  10:  
  11:             $.ajax({
  12:                 url: url,
  13:                 dataType: "json",
  14:                 type: "POST",
  15:                 data: '{ "p1": "Parameter1", "p2": "54321" }',
  16:                 contentType: "application/json; charset=utf-8",
  17:                 dataFilter: function (data) { return data; },
  18:                 success: function (data) {
  19:                     alert('success');
  20:                 },
  21:                 error: function (XMLHttpRequest, textStatus, errorThrown) {
  22:                     alert(errorThrown);
  23:                 }
  24:             });
  25:  
  26:         
</script>

   6:  

   7:     </body>

   8:  

   9: </html>

This small piece of JavaScript is very powerful when you quickly want to test your WCF service. First, reference the jquery.js file (via the Internet or a local URL).

jQuery.support.Cors = true is meant to enable cross domain scripting. But not always necessary.

Next, specify the URL of the method you want to use.

And then finally, perform the Ajax call. It’s important you use and correctly specify all these parameters. Make sure that the dataType and contentType are correctly set. Also make sure the type is POST as we specified this earlier. Next big thing is the data property. As you can see, the parameter names are exactly the same as in C#.

If the call succeeds or fails, separate JavaScript is executed.

You can also use cUrl to make quick calls to the service but I will not go deeper into this here.

- Use the IE9 developer tools

They allow you to monitor the data that is sent and received so you can immediately see if everything is OK there. Do this by visiting your test HTML file, press F12, go to the ‘Network’ tab, click ‘Start capturing’. Then reload the test page, get back to the test HTML file and take a look at the POST entry.

If you click it, you can see the sent and received data in respectively the ‘Request body’ and ‘Response body’ tabs.