Look alive. Here comes a buzzard.

First off, let me say that the AutoSPInstaller project is great. It’s awesome that we have a community driven effort underway to automate SharePoint 2010 installation and configuration. But I recently started building out a fairly large and complex SharePoint farm and wasn’t happy with certain aspects of the AutoSPInstaller script. Specifically, I wanted a better configuration for a multi-server environment and I wanted more control over what was installed where.

So I put together a new set of scripts called SPFarmInstaller.

The core of SPFarmInstaller is the Farm Configuration XML file.  Within this file you define everything about your farm.  This is also the first key difference between SPFarmInstaller and AutoSPInstaller.  With SPFarmInstaller you only have one farm configuration file.  This file is the same regardless of which server you are working on.

Within your Farm Configuration File you define:

• Basic farm information like the DB server, Config DB name, outgoing email settings, etc.
• Managed accounts to be used by your services and application pools
• Servers in your farm and which services are running on them
• Service applications such as Search, MMS, etc

I’ll provide more detail on the Farm Configuration file in a future post.

The next step is to copy the XML File and the SPFarmInstaller scripts to each server in your farm.  You will also want to make the SharePoint installation bits available to each server in your farm.

Next comes the second major change from AutoSPInstaller.  Rather than one installation script that does everything, SPFarmInstaller has each step of the installation in a separate script.  This gives you greater control over what you configure and when.  It also makes it easy to snapshot your servers at each step of the way if you are using virtual machines.

So, there are 4 major scripts that you will use:

• 00-download-prereqs.ps1 – this script is optional and will download the SharePoint prerequisite files to the local server
• 01-install-bits.ps1 – this script must be run on each server in your farm.  It will install the SharePoint bits (including prerequisites) but nothing else.  It will not configure or join a farm.  I like to run this script in parallel on all of my servers and then snapshot them so I have a clean installation of SharePoint I can roll back to if necessary.
• 02-create-or-join-farm.ps1 – this script is run first on your Central Admin server where it will create a new SharePoint farm and provision the Central Admin web app.  After that it is run on each of the other farms in your environment joining each server to the existing farm.  If you want you can stop now and continue configuring your farm by hand.  But why configure by hand when you can use the next script?
• 03-configure-farm.ps1 – this is the last script to run and it should be run from the Central Admin server only.  This script will set up your server topology (starting and stopping various SharePoint services on each server in your farm), create your web applications, and provision all service applications you have defined.

Check out the Read Me for detailed instructions on running these scripts.

As a warning, I wouldn’t consider this 100% complete just yet.  I’m using it for building out several production farms so I know the code works for my needs.  But I haven’t tested every configuration and option.

You can download the scripts here: https://github.com/glenc/SPFarmInstaller/zipball/master

Again, let me give full credit to the AutoSPInstaller project.  Most of the functions and code in my scripts have been pulled from that project or greatly inspired by their approach.

Out of the box, there is no way to copy a web part page and have it retain all of its web parts – including List View web parts.  I tried the following methods:

• Download .aspx from Pages doc lib, rename and re-upload.  All list view web parts are stripped during this process.
• Use the Send To option to make a copy of the .aspx in another folder or doc lib.  Same result as above – all web parts are stripped.
• Use the Content Structure view to copy the page.  Same result, web parts are stripped.

Finally I found a solution – although it’s a bit of a hack in my opinion: SharePoint Designer workflow.  The SPD workflow action “Copy List Item” will make a copy of a page and retain all web parts including list view web parts.  Here’s how you can accomplish this:

1. Open SPD, open your site, and create a new Workflow.  Call it “Copy Page” or something like that.
2. Target your Pages library (or wherever your web part pages live) for the workflow and specify that it should be started manually.
3. Add a single action:  Copy List Item.  When adding the list item, you can select “Current Item” for the page to copy, and then specify the target list as the same Pages library.

Now you’ll be able to start the Copy Page workflow on any page in your pages library.  When it creates a copy it simply appends the date and time to the file name so you won’t wind up over-writing the existing page.

With a little more work it should be possible to create an Initiation Variable for your workflow and rename the page as a part of the workflow.

One of my biggest complaints about SharePoint 2007 is that as a developer, it is virtually impossible to employ professional development techniques when building SharePoint solutions.  I’m talking about source control, continuous integration, test driven development, etc.  Obviously you can use these techniques when it comes to custom web parts, event receivers, and other “code only” solutions, but when you’re talking about larger projects that include content types, SharePoint Designer workflows, custom lists, in addition to custom code, it simply isn’t practical to try and cram everything into source control.  It’s just too much effort.

From what I’ve seen so far, many of these issues will be resolved with SharePoint 2010 and Visual Studio 2010.  I’m looking forward to digging into a real project so I can find out where the “gotchas” are (and I’m sure there will be plenty), but so far I’m optimistic.  I am hopeful that as a developer, I will be able to bring professional development techniques to all areas of a SharePoint project.

In my opinion this is critical if Microsoft wants to make SharePoint a true application development platform.

I’ve been wrestling with this error off and on for a while now.  It shows up in the event log of a production MOSS server I help manage.  Here is the error message:

Event Type: Error
Event Source: Office SharePoint Server
Event Category: Publishing
Event ID: 5165
Date: 8/4/2009
Time: 12:57:51 PM
User: N/A
Computer: SERVER
Description:
Console Configuration File Error: File Not Found: The system cannot find the file specified. (Exception from HRESULT: 0×80070002)
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Googling for this error provides some information, but nothing definitive.  After turning up the logging level for the Publishing category, I was finally able to find enough information to locate the root cause.

This problem occurs when you are using a Master Page referencing SharePoint Publishing controls in a site collection that does not have the Publishing Infrastructure feature activated.  In my case, I was using a Master Page with the Publishing Console control on the page.  This Publishing Console was looking for some files in the Master Page gallery (_catalogs/masterpage/EditingMenu/CustomEditingMenu.xml etc) which don’t exist if the Publishing Infrastructure is not activated.  After I activated the Publishing Infrastructure, the errors went away.

However, ultimately the real solution was to remove the publishing features from the Master Page since I wasn’t using them anyway.

This problem came about when we re-used the master page from our main intranet on users’ My Sites.  Since the My Site doesn’t use the publishing features, every time someone hit a my site, these errors would show up in the event log.

I wanted to do a quick post on why I chose Ruby for RPoint. I talked about this a little in the first post but just looking at the examples I’ve given so far, one could make the argument that rather than creating these scripts in ruby, you could just create a simple XML document that defines your structure, templates, etc.

This is very true – and in fact I have written tools like this in the past. But here’s the problem I’ve encountered with XML. You end up trying to build a programming language out of XML. That’s basically where CAML has ended up. And I don’t know a single person who actually enjoys looking at, much less creating, CAML.

Here’s what I like about the ruby approach. It let’s you write something like this:

site_names = ["Site One", "Site Two", "Site Three"]
at "http://localhost" do
  site_names.each do |name|
    create_web name, TeamSite
  end
end

Just the simple addition of an array of site names is something that would be hard to do in XML. What if you wanted to pull your information from a database? Or a file system? Or an Excel spreadsheet? For any of those you would have to write more logic into your XML mini-language – constructs like looping, conditionals, some mechanism to handle plug-ins for additional logic, etc. By leveraging an actual programming language, we get all of that for free.

Want to create 50 test sites? Here you go.

at "http://localhost" do
  50.times do |i|
    create_web "Site #{i}", BlankSite
  end
end

’nuff said.

Yesterday I posted about RPoint for the first time and I showed you a simple script to create a site structure. All of that code is working today. I wanted to keep the example simple and show you something that was actually functional.

Today I want to dream a little. Here are a few scripts I want RPoint to be able to support.

Creating a List Template

This script would create a list template.

class Milestones
  include ListTemplateMixin
  
  name "Milestones"
  description "A Simple Milestone List"

  column "Milestone", Text, :is_title => true, :required => true
  column "Description", RichText
  column "Date", Date, :required => true, :default => :today

  view "All Milestones" do
    show_columns "Milestone", "Date"
  end

  view "Overdue Milestones" do
    show_columns "Milestone", "Date"
    where :date => less_than(:today)
    order_by :date => :descending
  end
end

Once you had defined your template, you could use it in a few different ways. First, you could use it in a site creation script:

at "http://localhost" do
  create_list "My Milestones", Milestones
end

Second, and possibly cooler, you could generate a list definition:

>ir generate.rb list_definition --source MilestoneTemplate.rb

Migrations

Migrations in Rails is a great pre-defined structure for handling your database schema and changes to it over time. It may not be anything too ground-breaking on its own, but it enforces a discipline when it comes to managing your database. You are encouraged to think about situations where you want to change your database but data already exists. You can write scripts to migrate existing data to your new structure and it’s all managed within source control as an ordered list of steps to execute.

I want to be able to do something similar with RPoint.

# first migration - create my site collection
class CreateSiteMigration
  at :webapp => "http://localhost" do
    create_site "My Site", TeamSite
  end
end

# second migration - add a list and add a lookup to that list on an existing list
class AddIssuesMigration
  at "http://localhost/sites/mysite" do
    create_list "Issues", Issues
    update_list "Tasks" do
      add_column "Related Issue", Lookup("Issues", :field => :title)
    end
  end
end

Okay – that’s enough dreaming for now. Just incase anyone missed the intro THIS CODE DOES NOT WORK. It is completely imaginary. I am hoping that something along these lines will become reality. And I hope that you will agree that this would be a much nicer way to create SharePoint solutions than slogging through hundreds of lines of XML.

A few days back I talked about how the release of IronRuby would be great for SharePoint scripting. In my opinion, what makes Ruby such a powerful language is that it is great for creating domain specific languages (DSLs). I’m going to keep this post short so I won’t go into what DSLs are or examples of them. If you want to find out more info, look at some other ruby libraries such as rspec, rake, and datamapper.

I think that SharePoint is in dire need of better tools to facilitate solution development, deployment, and maintenance. It is simply too difficult to build a site in one environment and then deploy it in production. Sure you can do backup and restore, but what about when you want to make some upgrades?

Anyway, this weekend I started working on an IronRuby-based DSL for SharePoint scripting. I’m calling it RPoint for now (until I think of something sexier!). Now, this is very very very early in development and I am sure things aren’t set up today as nicely as they should be long term, but I’ve been really excited by the progress and the possibilities.

RPoint will allow you to write code like this to create SharePoint sites:

# create a new subweb beneath an existing site
at "http://localhost" do
  create_web "New Web", TeamSite
end

# create a new site collection and website structure
at :webapp => "http://localhost" do
  create_site "New Site Collection", BlankSite, "domain\\user" do
    inside root_web do
      
      # create our site structure
      create_web "Child Web", TeamSite
      create_web "Another Child", BlankSite do
        create_web "Private Site", TeamSite, :inherit_permissions => false
      end
      
      # create a few lists
      create_list "Documents", DocumentLibrary
      create_list "Announcements", Announcements

    end
  end
end

I think that’s pretty cool. Beyond simply creating sites, RPoint will allow you to create lists, views and probably upload resources like images and documents.

My goals for RPoint are to be able to do the following:

• Easily write scripts to create site structures and lists
• Rails-like “Migrations” for applying changes to existing sites
• Custom site template and list definitions in ruby as opposed to XML
• Ability to run template/list definitions against a live site or generate schema.xml/onet.xml files for production deployment
• Support web-services automation as well as object model automation (this will be tough)

SharePoint is a huge product. The code-base is enormous and the aspirations for RPoint are lofty. I would greatly appreciate any assistance. The more hands there are to help, the sooner we will have a tool to make SharePoint development a little easier.

To get RPoint in its current form, you can download it from GitHub.

I wrote a while back about SharePoint scripting with IronPython. Well today’s announcement of the first binary release of IronRuby got me pretty darn excited. I really like Python, but I LOVE Ruby. I’ve been waiting for IronRuby to catch up to IronPython for quite a while and it looks like they’re getting close.

Here’s a quick guide to getting up and running with IronRuby and SharePoint.

Installation:

1. First, download the IronRuby release from Rubyforge.
2. Next, copy the ironruby folder you extract to your Program Files (put it wherever you like – that’s where I put mine).
3. Update your system’s PATH environment variable to contain the path to the ironruby\bin folder.  So if you placed it in your Program Files, you would add this to your PATH: c:\program files\ironruby\bin

Creating a Script:

Next you’ll want to create a new text file called print_sites.rb. I recommend the e text editor by the way – great tool.  Add the following lines to your file:

require 'System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089, processorArchitecture=MSIL'
require 'Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c, processorArchitecture=MSIL'

Uri = System::Uri
SPSite = Microsoft::SharePoint::SPSite
SPWebApplication = Microsoft::SharePoint::Administration::SPWebApplication

Okay – what does that do? Well the first two lines are like adding a reference to a standard Visual Studio project. They tell IronRuby that we need to reference (require) the System and Microsoft.SharePoint assemblies. The next three lines are for convenience. They set up aliases to the System.Uri class and two SharePoint classes. In our ruby code, now we can just use Uri, SPSite, etc instead of having to type out the fully qualified path.

Next, let’s add some ruby code to enumerate all sites in a web application.

webapp = SPWebApplication.Lookup(Uri.new("http://localhost"))
webapp.Sites.each do |site|
  puts site.Url
end

(hmm, looks like I don’t actually need the SPSite alias after all)

Now, save your file. You can execute it like this:

> ir print_sites.rb

If everything is set up right, you should see the URL for each site collection be printed to the screen.

There you go. Have fun.

I wrote a small introduction to SharePoint scripting with Python a while back and this weekend I didn’t have anything better to do (well actually I did, but I’m an addict – what can I say) so I decided to expand on it.  I’ve started putting together a small Python library for working with SharePoint using IronPython.  It’s called – amazingly – sp.py.

So far the library if pretty small – although it does include a nice little module for workign with stsadm commands.  I’ll be adding to it as necessary.  I’ve decided that although adding custom stsadm extensions is nice and does fit into the “way to do things in SharePoint”, it’s just too much overhead when you want to do something simple.  So at this point when things come up that need automation, I’ll be adding new sp.py scripts.

In addition to the sp.py library I’ve added three scripts:

• applytheme.py – apply a theme to all webs in a web application
• backupsites.py – backup all site collections in a web application
• setprofileprop.py – set a property on all user profiles based on a dynamic pattern

So where can you get this awesomeness?  Well, actually you can get it on github.  Github???  You mean that totally web 2.0 site for hosting Git repositories?  Yeah – that’s the place.  If you’re wondering why I’m using git for source control, well I guess Subversion just wasn’t “edge” enough anymore.  Git’s actually pretty cool – and if you need to use Windows you can download a Windows version here.

At some point I’ll figure out how to post it as a zip or something instead of a tarball.  But until that point, you can grab the tarball here.

I’ve had this project in the works for a while now and it’s time to open it up to the world. The SharePoint Knowledge Base is a simple wiki site I created to capture some best practices, how-tos, tips and tricks, etc. While I think that blogs are great tools to communicate this sort of information, I’m not satisfied with the experience of searching for a specific issue or problem. Often times I find myself sifting through numerous blog posts – sometimes with conflicting information – left wondering which author knows his or her stuff.

The intent of the SharePoint Knowledge Base is to provide a wikipedia-like site dedicated to capturing community-driven expert content around SharePoint.

Today, there is not a lot of content. That’s where you come in. We all come across things every day that the rest of the community can benefit from. Why not take a minute and jot down your notes here in the SharePoint KB? Not only for your own future reference, but for the benefit of others as well.

There is no registration required to create or edit content. Until we start to have problems this site will remain completely open for anyone to edit.

So please, if you have something to contribute I would encourage you to post it here.  We can all benefit from a better way to store and update SharePoint-related information.  There’s so much of it out there but it’s so hard to sift through.  I hope that together we can start to chip away at this problem.