Designing a high quality SharePoint solution – how hard can it be?

Well, quite hard actually.

In my job, I am often engaged in design review and troubleshooting assignments, and I have met quite a few clients that have different issues with their SharePoint-based solutions, such as performance, bugs and maintainability issues. These issues unfortunately prevents them from utilizing the platform to its maximum potential and gives SharePoint a bad reputation both within and outside the organization.

In this post I would like to spend some time discussing what I believe are the criteria for a high quality SharePoint solution and some considerations that has to be made before moving on developing and deploying the solution to ensure a greater chance of success.

1. It just works!

The first obvious quality criteria is of course that the solution works as expected from a functional point of view – without bugs that prevents users from performing their daily tasks. This is true for all IT solutions, independent of the underlying platform. In addition to skilled developers and product specialists, you need a proper QA process to ensure this. If the project budget or timeframe doesn’t allow for proper testing, the scope of the project should be cut or you might even consider if the project should be carried through at all.

However, in my opinion, there is more to quality than just having a solution that works as expected.

2. The solution solves the business problem in a cost effective way

While SharePoint provides a fantastic array of features and services out of the box, it can be quite tough sometimes to adapt it to requirements that are not quite in line with the standard functionality. Designing a solution that solves both the business requirements, while being cost effective and maintainable is quite often a challenge.

An important key to making the right design decisions is really try to understand the problem you’re trying to solve for your client, not just reading and following the requirements specification exactly. In SharePoint there are often many ways to achieve the same goal and as a solution designer, you should always try to find an approach that solves the clients problem and at the same time is cost effective to develop and maintain. As some wise person have said before – the requirements are always open for negotiation.

SharePoint has a huge amount of features and services built into the platform – good knowledge of these and knowing how to apply them for solving different business challenges is crucial. It’s equally important to know the limitations of SharePoint – it’s not the right choice for everything!

3. The solution is easy to maintain, develop and troubleshoot

The flexibility of SharePoint can actually become quite an issue once the solutions has been deployed and up and running in the production environment. While customizations with SharePoint Designer can be made without much effort, doing proper QA and moving the changes from a test to production environment is not as straight forward as one could hope for. It’s very important to keep these issues in mind when defining the policies for how and when changes should be made.

In my opinion, the way Microsoft is marketing and selling the SharePoint platform can be a bit problematic sometimes. Need to pimp up the look and feel of your intranet? Easy, just fire up SharePoint Designer and do it! Sounds great, eh? Who wouldn’t want to be less dependent on those expensive consultants and specific developers who are the only ones that knows how to do even the simplest  change in the systems?

As many of us are painfully aware of, the reality is not quite that simple, at least if you consider your application to be critical or at least somewhat important for your business. If that is the case, doing changes directly in your production environment without a proper QA and change management process is just unacceptable. I certainly hope that Microsoft provides us with better guidance and better tools for moving changes and customizations between environment in the next version of SharePoint.

What needs to be done before every SharePoint solution is deployed to production is to find the right balance between flexibility and manageability. What changes needs and can be delegated to site owners to perform and what needs to be done by developers? And how does the change management process look like in your specific organization? There need to be very clear guidelines for this in place, and the right way to go depends on many factors, such as how business critical the application is, the amount of custom code that depends on i.e. certain columns and lists being in place, where the responsibility lies if something goes wrong and end user knowledge of the platform and tools.

Hopefully, there is a governance plan in place that addresses at least some of these issues.

And also, if your solution consist of a fair amount of custom code, good coding standards and design principles needs to be applied, as in any type of development project.

4. The solution is easy to upgrade to future versions of SharePoint

The upgradeability factor is always important to keep in mind, but it’s becoming more and more important as the next version of SharePoint (SharePoint 15) comes closer to public beta.

In general, the SharePoint Team puts a lot of efforts to ensure the backwards compatibility of the platform. Upgrading an out of the box size or a site with a few customizations to the next version of SharePoint is often a very straightforward process (at least from my experience from upgrading solutions from 2007 to 2010). UI customizations and custom developed components will add additional effort though. This is particularly true for CSS “hacks” on top of the out of the box styling, so it’s wise to minimize this as much as possible if an upgrade to the next version of SharePoint is on the roadmap during the expected lifetime of the application.

Make sure that all customizations and changes in your SharePoint environment are supported and in line with best practices.

5. The solution has a low impact on the SharePoint farm

It’s important to keep in mind that SharePoint farms are shared between many applications and sites and typically grow significantly as more and more users adopt the platform. When building full trust solutions, it’s extremely important to make sure that your code performs well and has a low impact on the shared server resources so it doesn’t risk sinking down performance for other applications hosted in the farm. Sandboxed solutions are less of a problem, since they run in isolation and the resource usage is limited by quotas, but the limitations of the sandbox often forces developers to go for a full trust solution.

Performance testing is the key to ensuring this – it’s basically the only way to know for sure that your solution performs well in a real world scenario. Take your time to set up and run scenarios with both realistic load and volumes of data. It’s well worth the time to learn and master a good performance testing tool such as that provided with Visual Studio.

During the performance tests, make sure to observe the CPU and memory usage of both the SharePoint and SQL Servers to ensure the CPU usage is ok and there are no memory leaks. You should always aim for good response times, as well as an impact on the farm that’s low enough for allowing growth over time in both usage and number of applications. If the results are not good enough, consider performance optimization and/or scaling up the farm before deploying to production.

Wrap up

Though many of the points above may seem a bit obvious, there are many factors to keep in mind when designing SharePoint solutions. The ideal development team needs knowledge in business process analysis, managing requirements, deep knowledge of the features and services in the SharePoint platform, infrastructure, coding skills, functional and performance testing skills, good command of HTML, CSS and JavaScript and much more. There is no superman that can possibly be an expert in all these areas so it’s important to make sure that the project team consist of a good mix of people that covers as much of the needed skills as possible.

I hope that some of you might find these points helpful, and hopefully don’t intimidate you too much to refrain from taking on the challenge. Building successful SharePoint solutions might be challenging, but it’s well worth the effort to try to get it right when considering the great impact a good solution can have on your business!

Naming conventions for SharePoint 2010 environments

[Updated 2011-12-29]
I have updated the guidelines below after some good feeback in the comments.

In my job as a SharePoint Architect, I frequently help different clients setting up their SharePoint environments and helping out with operations and troubleshooting. When operating a SharePoint environment, a consistent naming of SharePoint objects such as web applications, application pools, databases and so on definitely makes life a lot easier. There are no official guidelines regarding how to name things though – administrators are mostly free to give the objects any name they like – but it’s definitely a good practice to have some form of consistency here.

Below is my proposal for naming conventions that should work well for most organizations. It’s not something I have invented myself – it’s rather a compilation of how most clients I have worked with choose to name their SharePoint objects.

Web Applications

Web Applications should be given a name that clearly reflects both the default URL and port number.

Proposed pattern: {Host Header} – {Port}

Example for a site with host header intranet.contoso.com, port 80:
intranet.contoso.com – 80

Service Applications

The most straightforward way to name service applications is to use the same name as the service application type. You can see the names of the different service application types in the “New” dropdown button on Manage Service Applications in Central Administration.

If more than one service application exists, just add a dash and an additional value that differentiates the service application.

Examples:
Excel Services Application
Managed Metadata Service
Secure Store Service

Examples with more than one service application of the same type:
Corporate level: Managed Metadata Service
Department level: Managed Metadata Service – Sales Department

Application Pools

For web applications and service applications with their own application pools, simply give the application pool the same name as the web application or service application. For shared application pools, give it a name which clearly reflects how it’s used.

Databases

The database names should have a clear relationship to the web- or service application it belongs to. Also, it’s good to not go too far away from SharePoints default naming suggestions. Without the nasty GUIDs of course.

Option 1 – Web application and service application name
The following option works well if you want a naming convention that’s similar to SharePoints default names, and where each farm has a separate SQL instance.

Proposed pattern for Content Databases: WSS_Content_{URL to Web Application}, dots converted to underscore.
Example: WSS_Content_Intranet_Contoso_Com

Proposed pattern for Service Applications: {Service Application Name}, blankspaces converted to underscore.
Example: Managed_Metadata_Service

If needed, databases could also be prefixed with a value that differentiates the databases. This makes management easier since databases are listed in alphabetical order in SQL Server Manager.

Option 2 – Technology_Organization_Scope_DatabaseType
John W Powell has proposed this pattern for naming database which works well for environments where several farms and organizations share the same database instance. However this system for naming databases differs quite a lot from SharePoint default names and may feel unfamiliar for many admins. And since all databases are prefixed with the specific SharePoint version number, a lot of renaming will be needed after future upgrade to the next version of SharePoint.

http://blogs.msdn.com/b/johnwpowell/archive/2010/11/20/sharepoint-2010-database-naming-standards.aspx

Solution Packages

When building your solution packages, you can let your build server add a label to the WSP packages before putting them at the drop location. This makes it easy to see exactly which version is installed in each environment though the Manage Farm Solutions page in Central Administration.

Proposed pattern: {Company}_{Solution Name}_{Version}

Example: contoso_intranet_1.0.0.23.wsp

Other objects

In addition to these naming conventions, there are several other SharePoint objects such as service accounts in Active Directory and server names, where  naming conventions usually applies. Most organizations already have a system in place for naming these objects that needs to be followed, independent of SharePoint.

Share your thoughts and ideas!

What do you think of these proposed naming conventions? Got any additional ideas or alternative options? Feel free to post a comment with your suggestions.

Design Patterns for developing SharePoint 2010 Solutions

Last week at the European SharePoint Conference, I talked about a few different design patterns that you can use when you develop custom solutions for SharePoint 2010, to make sure that your code is easy to maintain, extend and test.

We specifically talked about the following patterns:

• Repository
• Model-View-Presenter
• Service Locator

Each of these patterns offers a solution to different design issues that you might encounter when creating code based solutions. Our development team has successfully applied these patterns on several client projects where a fair amount of custom development was needed. However, applying the patterns will require some initial extra work and requires the developers in the team to be somewhat familiar with the patterns. So only apply them when you really see that you can get some benefit out of them.

Please check out the slides and sample code from the session.

To get started with the sample solution, please do the following:
• Download and unzip the package
• Open the SharePointEurope.DesignPatterns4 solution (the final solution where all patterns have been applied, as well as the configuration settings management)
• Deploy the SharePointEurope.Dependencies package to ensure that the common service locator library and application foundations are deployed to the GAC
• Create a site collection and deploy the SharePointEurope.DesignPatterns4 project to the site.
• Generate a Google Map Key for the host name of your web application and go to the configuration list (/Lists/Microsoft_Practices_SharePoint_ConfigList) and set this value.
• Create a few hotels in the Hotel list (/Lists/Hotels)
• Add the hotel finder web part somewhere and try it out

A few notes regarding the things we discussed at the session:

Repository implementation
There are many different ways to implement the repository pattern in SharePoint. In my sample implementation you have two options when creating the repository: Either provide the repository with a specific SPWeb object in the constructor, or use the context of the current web site. It’s important to make sure that you can use your repository from contexts where SPContext.Current is not available, such as i.e. timer jobs. Another option is to use configuration settings to provide an URL to a centrally located site where the hotels list is located – which is useful if you want to use the repository on other sites than where the list exists.

If you look at how the training management application in SharePoint Guidance 2007 is implemented, you can see that they use a slightly different approach. In that implementation, you need to provide the repository with an SPWeb object in each of the repositorys data access methods.

The problem with this approach is that your repository interfaces becomes cluttred with specific dependencies to SharePoint. An important goal of the repository pattern is that you should make yourself independent of exactly how the data is stored. And if you decide to change the data storage from SharePoint to i.e. SQL Server, you would need to not only change the implementation of the repository, but the interfaces and all the code that depends on it as well. In my opinion, it’s much better to let the repository get the data context it needs internally instead, or to use dependency injection to configure exactly how it’s provided to the repository. Here you could benefit quite a lot from using a proper dependency injection framework rather than the SharePoint Service Locator.

List based configuration
During the demo I showed you a simple example on how you could generate a configuration list and use it to store, retrieve and update settings. If you decide to go for the list based approach, I encourage you to check out the List Based Configuration Quickstart that’s available as a part of the SharePoint Guidance 2010 package. It’s a bit more refined and basically implements the same interfaces as the property bag-based configuration classes do. This makes it possible to use lists to store you configuration settings for any level of the farm (farm, web app, site collection or web) by specifying the path to a centrally located site collection where the settings are stored.

Using Application Foundations in sandboxed solutions
I got a few questions after the session about using the Application Foundations Library in sandboxed solutions.

The application settings manager can be used in sandboxed solutions, but there are a number of limitations of course. You can read more about using the application settings manager in sandboxed solutions here: http://msdn.microsoft.com/en-us/library/ff798449.aspx

I haven’t tried the SharePoint Service Locator in a sandboxed solution yet – if anyone has done that, I would be happy to hear about your experiences. Otherwise, there are plenty of other options for implementing the pattern, including implementing your own simple code-based service locator.

Sweden SharePoint User Group in Stockholm

Last Thursday, Sogeti hosted a Sweden SharePoint User Group (SSUG) meeting in Stockholm. SSUG is a user group which arranges meet-ups a few times each year, where participants can exchange their thoughts and experiences on SharePoint.

The first speaker was Markus Persson from Sogeti SharePoint Center in Stockholm. He talked about and have a demo of a Business Intelligence portal that has been built on SharePoint 2010 for the Municipality of Umeå in Sweden. The solution gives executives on different levels in the organization quick and easy access to the statistics and key performance indicators that are relevant for them.

Another speaker was Peter Nicks, who is Product Marketing Manager for SharePoint in Sweden. He gave a presentation about how to leverage the power of SharePoint within the organization, and how to take advantage of all the features and services that the platform provides. Måns Barklund from Microsoft also held a presentation and demo of Project Server 2010.

SharePoint User Group is a great forum for everyone who works with SharePoint – developers, end users, administrators and IT Pros. More information and upcoming events can be found at the web site www.ssug.se

A case study of the the BI solution of Umeå can be found here.

 

SharePoint 2010 Genesis Framework – A framework that will help you create amazingly structured SharePoint solutions

Summary: SharePoint 2010 Genesis Framework is a framework that helps you declare fields, content types, lists and feature definitions through code. It also helps with provisioning and stronly typed data access.

One of SharePoints greatest strengths is its flexibility. Power users can easily create very powerful solution without much involvement of developers. In many cases though, custom development is needed for parts of a solution. In those cases, it’s very important to properly package the code together with the storage structures such as site columns, content types, lists, views and so on into a solution package.

However, doing this the “classic” way – defining the storage elements with CAML can cause some challenges both during the development phase and when the solution goes to application management. Most developers struggle with the following challenges:

• A lot of CAML/XML needs to be written
• It’s difficult to handle additions and changes to content types already in use
• No strongly typed access to SharePoint Data, unless you choose use LINQ-2-SharePoint

The new upgrade actions in SharePoint 2010 helps somewhat, but when working in an agile manner with deployments every 2-3 weeks, the XML can get really messy.

What many developers do is to manage most of the provisioning of content types, site columns, etc. with code and “ensure-style-features” that incrementally update the content types, lists, views, etc. This requires quite a lot of coding though, and many developers create their own helper classes that quickly evolve into “mini-frameworks”.

Tony Restaino, a senior SharePoint developer at Avega has been working on this issue for a while and have recently released an extensive framework to solve this issue – SharePoint 2010 Genesis Framework (SPGEN).

Tony and I have been working for a client on a two large SharePoint projects for a while now, and we have been using this framework to define content types, site columns, features, etc. and all the provisioning and data access. Want to add a new field to a content type? No problem – just create the field as a class, add it to the content type, rebuild and deploy the code, reactivate the feature and it’s there! The framework has also helped us implementing the repositories with with strongly typed data access in no time!

It’s just amazing how easy life gets with solutions like these! If only Microsoft could be just as thoughtful as Tony when developing the SharePoint API🙂

A few of the key features of SPGEN:

  • It allows you to create classes that represent the different kind of elements in SharePoint – Site Columns, Content Types, List Instances, Views, Features etc.
  • You set attributes on the classes that correspond to the attributes of the SharePoint XML elements. This gives you strongly typed access to the definitions at any time!
  • The framework provides a lot of helper methods to provision fields, lists, views, etc. It automatically takes care of incremental updates, such as adding a new field to a list.
  • It also provides a way to easily create ListItem–object field mappers that maps list items to any class – no need to use the nasty auto generated mega classes from SPMetal!  You can even create your custom field adapters if you have custom columns that maps to complex data types.
  • It provides data access methods that allows you to do both CAML and LINQ queries against the lists, and returns the result as strongly typed objects.
  • It provides a LINQ implementation that has a LOT better performance the LINQ-2-SharePoint (at least according to our own measurements).

As you can see, it’s a very competent framework that can save a you lot of work – especially when developing custom code-based solutions. I strongly encourage everyone to check it out and give it a try!

Read more and download the framework here:

http://spgenesis.codeplex.com/

Configuration Management with Sandboxed Solutions

In our current project, we’re just about to introduce our first sandboxed solution into our SharePoint farms. This raises some interesting questions regarding how this fits into our existing configuration management process.

Until recently, we only had full trust solutions in our farms. Do deploy these, we go through a pretty straightforward change management process: The build server builds the solution packages, labels the code and assigns the version number to the WSP name. Developers can then deploy the packages to the integration environments, to which they have full access as administrators. When the packages are ready for testing, developers sends a request to Configuration Management (CM) to deploy certain packages to the test environment. CM then contacts operations, who perform the deployment. When user acceptance testing is completed, the packages are deployed to production.

This process works very well for full trust solutions. But how do things change when we start introducing sandboxed solutions? It’s not obvious at first how these fit into the established configuration management process.

If the user code service is enabled, sandboxed solutions can be deployed to any site collection without involvement of the farm administrator. It’s considered safe to do so because the code runs in an isolated process, and because the resource usage is monitored and limited. Does this mean that should be free for all site collection administrators to install whatever solutions they want on their sites?

The answer is, of course, no. The sandbox is just an execution model that affects the maintainability and the level of isolation the code runs in. However, from a configuration management point of view, sandboxed solutions are no different than full trust solutions.

Sure, the sandboxed solution can’t compromise the overall stability and performance of the farm, but it can still mess up the site collection it runs in. If the code you write provides features that are business critical for the end users (or might become in the future), it doesn’t matter if you run it in a sandbox or not. If you want predictable results, you still need to test your solution in a production-like test environment. And of course, you still need to keep track of exactly which version is deployed to each environment.

So, we decided to apply the same CM process to all solutions – sandboxed or not. The only difference is that operations doesn’t need to perform the deployment, site collection admins can do that themselves after getting approval from CM. And of course, all packages must go to the test environment first to ensure everything works as expected before deploying to the production environment.

E-mail management in SharePoint – why third party software is required

I have recently been working with a case management solution for a client. One of the most important requirements was that the e-mail conversations related to the case needed to be stored in a structured way that allows other people in the organization to access it if needed. The content of the e-mails also needs to be searchable, which the SharePoint search engine will do out of the box.

Even though SharePoint has a quite sophisticated integration with Outlook, it’s not as easy as one might think to move incoming and outgoing e-mail messages from the user’s personal inboxes to SharePoint. Sure, you can connect a document library in SharePoint to Outlook, but that will only give you read access to the files. If you want to store e-mails in a document library you need to use the explorer view or upload the msg files manually. That’s just not good enough if you want the end users to really adopt the tool and not just continue as usual, storing e-mails in their own exchange folders!

After trying out a few different options, we decided to go with a third party add in to Outlook called Colligo Add-In for Outlook. This component allows you to connect SharePoint libraries to Outlook and work with them just as folders in your mailbox. It also automatically extracts metadata about the e-mail such as subject, recipient, sender, etc and stores it in corresponding SharePoint columns. The possibility to just drag and drop e-mails to from the mailbox to SharePoint makes a world of difference for the end users.

There are a few other third party components available as well, but we decided to go with Colligo mainly because it’s a client solution that doesn’t require any server side component at all in the SharePoint environment.

Of course, the best thing would be if Microsoft could refine their Outlook integration a bit, and provide this functionality out of the box. Until that happens, we’re really happy that there are plenty of third party software vendors that can provide solutions for SharePoint’s weak points.

Follow

Get every new post delivered to your Inbox.