Look alive. Here comes a buzzard.

I’ve been having a lot of fun recently learning about and playing with RavenDb.  It’s a great document database solution for .NET.  The thing that immediately blew me away is how little friction there is getting up and going.  I am writing simple POCO objects and persisting them without having to deal with ORM mapping and other headaches.

However, there are a lot of naysayers out there spreading FUD about document databases.  Why would you need to move away from a relational model that enforces relationship constraints, allows for joins, and supports advanced querying and reporting??

Now, it’s perfectly valid to point those issues out.  If you are thinking of moving to a document database you need to know what you are sacrificing.  BUT – let’s talk about the benefits for just a moment.

Document databases tend to be a closer map to your domain model than relational databases.

Consider the scenario of persisting a customer:

public class Customer {
  public string Name { get; set; }
  public Address ShippingAddress { get; set; }
  public Address BillingAddress { get; set; }
}

In a relational model you will likely have a Customers table and a CustomerAddresses table with references between the two for the addresses. In the DB you will need to make sure the foreign keys are set up correctly and you will probably want deletes from the Customers table to cascade to the CustomerAddresses table. Then you will probably want to use an ORM like NHibernate to map those tables to the customer entity itself.

Think about that for a moment – how much work did we just have to do just to persist a simple customer?? And the reason we had to do that work is because we have to transform our data from the format WE want it in – the domain model – to a format that our relational database understands – tables and columns. That transformation takes real effort and thought. And it continually bites us in the ass with select n+1 scenarios.

I think if we step back for a moment and forget all our history with relational databases and really look at the problems we are trying to solve, it will be easier to choose the best solution for our system. Just because we’ve been building applications with relational databases for the past 20+ years doesn’t mean it’s the only way – or even the best way to go.

Sure with a document database we lose some of the relational power that something like SQL provides. But I believe the gains in many systems could overshadow those losses. And if we are designing our documents well then we won’t need as many relationships as we might have needed in a relational model.

First off, let me say that the AutoSPInstaller project is great. It’s awesome that we have a community driven effort underway to automate SharePoint 2010 installation and configuration. But I recently started building out a fairly large and complex SharePoint farm and wasn’t happy with certain aspects of the AutoSPInstaller script. Specifically, I wanted a better configuration for a multi-server environment and I wanted more control over what was installed where.

So I put together a new set of scripts called SPFarmInstaller.

The core of SPFarmInstaller is the Farm Configuration XML file.  Within this file you define everything about your farm.  This is also the first key difference between SPFarmInstaller and AutoSPInstaller.  With SPFarmInstaller you only have one farm configuration file.  This file is the same regardless of which server you are working on.

Within your Farm Configuration File you define:

• Basic farm information like the DB server, Config DB name, outgoing email settings, etc.
• Managed accounts to be used by your services and application pools
• Servers in your farm and which services are running on them
• Service applications such as Search, MMS, etc

I’ll provide more detail on the Farm Configuration file in a future post.

The next step is to copy the XML File and the SPFarmInstaller scripts to each server in your farm.  You will also want to make the SharePoint installation bits available to each server in your farm.

Next comes the second major change from AutoSPInstaller.  Rather than one installation script that does everything, SPFarmInstaller has each step of the installation in a separate script.  This gives you greater control over what you configure and when.  It also makes it easy to snapshot your servers at each step of the way if you are using virtual machines.

So, there are 4 major scripts that you will use:

• 00-download-prereqs.ps1 – this script is optional and will download the SharePoint prerequisite files to the local server
• 01-install-bits.ps1 – this script must be run on each server in your farm.  It will install the SharePoint bits (including prerequisites) but nothing else.  It will not configure or join a farm.  I like to run this script in parallel on all of my servers and then snapshot them so I have a clean installation of SharePoint I can roll back to if necessary.
• 02-create-or-join-farm.ps1 – this script is run first on your Central Admin server where it will create a new SharePoint farm and provision the Central Admin web app.  After that it is run on each of the other farms in your environment joining each server to the existing farm.  If you want you can stop now and continue configuring your farm by hand.  But why configure by hand when you can use the next script?
• 03-configure-farm.ps1 – this is the last script to run and it should be run from the Central Admin server only.  This script will set up your server topology (starting and stopping various SharePoint services on each server in your farm), create your web applications, and provision all service applications you have defined.

Check out the Read Me for detailed instructions on running these scripts.

As a warning, I wouldn’t consider this 100% complete just yet.  I’m using it for building out several production farms so I know the code works for my needs.  But I haven’t tested every configuration and option.

You can download the scripts here: https://github.com/glenc/SPFarmInstaller/zipball/master

Again, let me give full credit to the AutoSPInstaller project.  Most of the functions and code in my scripts have been pulled from that project or greatly inspired by their approach.

I’ve been playing with FubuMVC for about a week for a new project I’m working on (more to come on that soon).  So far I really like it, but it’s definitely not for everyone.  I thought I would start blogging about some of my experiences learning the framework.

There will probably be a few things I’m wrong about or don’t understand because I’m still new to FubuMVC.  But I want to capture my thoughts and impressions as somebody just starting out to help others who may be in a similar situation, or are evaluating the framework for their own projects.

Good: Conventions Everywhere

I think this is where FubuMVC really shines.  If you can define the conventions for your application up front then FubuMVC is going to save you a lot of time.  For example, FubuMVC lets you define an HTTP Method convention which can automatically apply to all of your controller actions.  You could define this convention based on method name (e.g. all Create, Update, or Delete methods only accept POST), input model type, or just about anything you want.

Here’s a snippet of code from my current application. It specifies that all controller methods which accept an input model ending with “Command” require the POST HTTP method.

This is a bit like Aspect Oriented Programming in that you can define cross-cutting rules which are defined once and applied everywhere by the framework.

Routing/URL Conventions
I really love the way routing/URL conventions are defined in FubuMVC. When working in ASP.NET MVC I find myself hard-coding a lot of routes to get them looking how I want. Defining routes in Fubu is quite a bit different from ASP.NET MVC, but I was able to configure my routing very quickly. FubuMVC also has a really nice diagnostics application that lets you see all routes you have defined for the entire application.

HTML Conventions
I haven’t dug into the HTML conventions yet but on the surface they seem very powerful. For example, you can define a convention where CSS classes are automatically added to lists for even/odd items. This is the kind of thing that should make it easy to apply cross-application UI changes easily.

In practice I think this will be a bit like CSS where as long as you create consistent HTML, style sheet can save you a lot of time in layout and formatting.

Bad: Sooooo many Models!

Okay this one took some getting used to. FubuMVC uses a “One Model In, One Model Out” (OMIOMO) approach for controller actions. That means that each controller action can only accept one model as a parameter, and returns a single model back to the framework. In ASP.NET MVC you might have a controller action that takes one or more simple parameters and returns an ActionResult, but in Fubu you must encapsulate everything into models.

This is the kind of thing that will turn off a lot of potential users. It looks “enterprisey” and feels like overkill at first. I didn’t want to create a ton of view and input models for my controllers so I tried creating generic models that would be used by many actions. For example, I created an EntityRequest model which had a single Id property that I could pass into all of my View, Edit, and Delete controller actions.

This worked at first, but in the long run it made the UI code a lot harder and more complicated. Creating links or form targets in FubuMVC is really easy if you can just give it a model. Honestly I’m not even sure how to construct a link to a controller action method.

In the long run I decided to simply create models for my different types of actions. And although this is quite a bit different from classic ASP.NET MVC, I actually like it better. It lets me de-couple the UI from the back-end. The UI simply says “give me the link to some model” and Fubu gets to figure out what actually handles that model.

Good: Moving away from Controllers

Once I accepted the OMIOMO approach I decided to make another change and do away with my controllers. Instead I will be adopting a Command/Query approach with Handlers on the back end to handle each command or query. I like this because my handlers can be a bit more generic and not tied so much to the UI.

Now even if you don’t want to move away from a Controller model, I think the real power here is that FubuMVC will still work just fine even if you want to adopt a different architecture.

Bad: Limited support for other view engines

Unfortunately the choice of view engines is limited to ASP.NET Web Forms and Spark. That being said, the ASP.NET view engine is pretty close to standard ASP.NET MVC. But I’d really like support for Razor or NHaml. I’m sticking with ASP.NET for now and it’s not that bad.

Bad: Documentation is still sparse

This comes with any open source project to a certain extent. But FubuMVC is pretty lacking in documentation at the moment. The getting started guides will only take you so far. Once you start trying to build your own application you will likely hit the limits of what you can learn from the guides themselves.

Another thing that complicates things is that FubuMVC went through a reboot a while back so there are some blog entries out there that are out of date with the current version. Many of the ideas in those posts are still valid, but syntax is slightly different in many cases.

Ultimately I’ve relied heavily on the FubuMVC source itself to figure out how to do things. For anyone trying to build a non-trivial application with Fubu it’s probably a requirement.

Final Thoughts

Lastly I thought I would compare FubuMVC to other frameworks/approaches out there. In many ways FubuMVC can feel quite a bit heavier and more “enterprisey” than other frameworks. But I really think it depends on the type of application you are building and what point in the application lifecycle you are at.

When you are just starting out, FubuMVC can feel very heavy getting everything set up. If you are building a small application with little functionality, you are probably better off with WebMatrix or ASP.NET MVC. WebMatrix especially gets you up and running very very fast.

But as your application grows and becomes more complex, FubuMVC should actually start to feel lighter and more flexible than those other frameworks. I’ve had many experiences with ASP.NET MVC where a change in design requires you to touch many different controllers, views, etc. If you are following your conventions these kinds of changes should be much faster and lighter in FubuMVC. At least that’s my impression at this point.

Perhaps a good approach is to start with something super light like WebMatrix and get a fast prototype up and running. Once your application starts to take shape and you can better visualize your conventions it might make sense to jump over to FubuMVC for the final implementation.

So far I am really enjoying FubuMVC even with all of the ups and downs. I’ll keep blogging about my experience building a non-trivial application with Fubu.

I’m starting out with FubuMVC and had a heck of a time resolving a strange error.  Whenever I would try to navigate to the _fubu diagnostics area it would just give me a 404.  It took me quite a while to find the problem because I’m still very new to FubuMVC and couldn’t figure out where in my configuration the error was originating from.

Turns out one of my controllers didn’t conform to the “One Model In One Model Out” rule.  I had created a controller method that accepted a single int id parameter.  FubuMVC didn’t like this and wanted a full model.  I was hoping it would consider a single parameter as a model even if it wasn’t a full class.  Once I converted my methods to the OMIOMO pattern everything started working again.

I heard about Nu on the Herding Code podcast and was immediately interested.  Ruby Gems is a great way to distribute open source libraries and manage dependencies.  The .NET scene has been struggling with this issue for quite a while so it’s great to see somebody take on this issue.

Not only does Nu make it super simple to add existing libraries to your project, but it’s also a great resource for finding existing open source projects out there.

I just happened to be building a brand new system so I’ll go through the steps I took to get Nu up and running.

1. Installed Ruby for Windows using the latest Ruby Installer
2. Opened a command prompt and ran the following commands:
   
   c:\gem update --system
   c:\gem update
   c:\gem install activesupport
   c:\gem install nu

3. Next I navigated to a new project folder and typed the following commands to add NHibernate and FluentNHibernate:
   
nu install nhibernate
nu install fluentnhibernate


That’s it – super simple!

Out of the box, there is no way to copy a web part page and have it retain all of its web parts – including List View web parts.  I tried the following methods:

• Download .aspx from Pages doc lib, rename and re-upload.  All list view web parts are stripped during this process.
• Use the Send To option to make a copy of the .aspx in another folder or doc lib.  Same result as above – all web parts are stripped.
• Use the Content Structure view to copy the page.  Same result, web parts are stripped.

Finally I found a solution – although it’s a bit of a hack in my opinion: SharePoint Designer workflow.  The SPD workflow action “Copy List Item” will make a copy of a page and retain all web parts including list view web parts.  Here’s how you can accomplish this:

1. Open SPD, open your site, and create a new Workflow.  Call it “Copy Page” or something like that.
2. Target your Pages library (or wherever your web part pages live) for the workflow and specify that it should be started manually.
3. Add a single action:  Copy List Item.  When adding the list item, you can select “Current Item” for the page to copy, and then specify the target list as the same Pages library.

Now you’ll be able to start the Copy Page workflow on any page in your pages library.  When it creates a copy it simply appends the date and time to the file name so you won’t wind up over-writing the existing page.

With a little more work it should be possible to create an Initiation Variable for your workflow and rename the page as a part of the workflow.

I’d like to take a few moments for some shameless self promotion.  Yesterday I was happy to release Sublime 1.0.  Sublime is a Subversion management application designed for Microsoft-centric environments.

Subversion is an excellent SCM tool – arguably the leading SCM tool available today. But with its roots firmly in the open source / linux community, there aren’t a lot of Windows-based applications for managing subversion.  Tools like VisualSVN server make it easy to install and configure Subversion, but it requires access to the server for management.  Web-based Subversion apps can be found, but they are often built in PHP and may not fit with a “Microsoft Shop’s” IT strategy.

Sublime was designed for exactly these reasons.  Sublime provides a web-based management tool for your Subversion infrastructure.  It allows users to browse and create repositories, manage permissions, and enable email alerts all from a simple web interface.  Server administrators can configure automated backups, manage users and groups, and monitor server health all from a web browser.  The Sublime installer will automatically install Subversion if required making the setup process extremely fast and simple.

Ultimately, Sublime makes it easy for dev teams to use Subversion without any bottlenecks.  Users don’t have to wait for IT to create repositories, users can manage their own password, repository owners can manage permissions, etc.  At the same time, IT has the ability to manage the server and ensure data is backed up all within a Windows-based environment they are familiar with.  Everybody’s happy

I’m super excited about this release.  If you use Subversion today, or are thinking of moving to Subversion from a different SCM tool (or *gasp* no SCM tool), then I would encourage you to take a look at the Sublime website for more information.

Okay.  Plug over.  We now return you to your regularly scheduled blog.

One of my biggest complaints about SharePoint 2007 is that as a developer, it is virtually impossible to employ professional development techniques when building SharePoint solutions.  I’m talking about source control, continuous integration, test driven development, etc.  Obviously you can use these techniques when it comes to custom web parts, event receivers, and other “code only” solutions, but when you’re talking about larger projects that include content types, SharePoint Designer workflows, custom lists, in addition to custom code, it simply isn’t practical to try and cram everything into source control.  It’s just too much effort.

From what I’ve seen so far, many of these issues will be resolved with SharePoint 2010 and Visual Studio 2010.  I’m looking forward to digging into a real project so I can find out where the “gotchas” are (and I’m sure there will be plenty), but so far I’m optimistic.  I am hopeful that as a developer, I will be able to bring professional development techniques to all areas of a SharePoint project.

In my opinion this is critical if Microsoft wants to make SharePoint a true application development platform.

I’ve been wrestling with this error off and on for a while now.  It shows up in the event log of a production MOSS server I help manage.  Here is the error message:

Event Type: Error
Event Source: Office SharePoint Server
Event Category: Publishing
Event ID: 5165
Date: 8/4/2009
Time: 12:57:51 PM
User: N/A
Computer: SERVER
Description:
Console Configuration File Error: File Not Found: The system cannot find the file specified. (Exception from HRESULT: 0×80070002)
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Googling for this error provides some information, but nothing definitive.  After turning up the logging level for the Publishing category, I was finally able to find enough information to locate the root cause.

This problem occurs when you are using a Master Page referencing SharePoint Publishing controls in a site collection that does not have the Publishing Infrastructure feature activated.  In my case, I was using a Master Page with the Publishing Console control on the page.  This Publishing Console was looking for some files in the Master Page gallery (_catalogs/masterpage/EditingMenu/CustomEditingMenu.xml etc) which don’t exist if the Publishing Infrastructure is not activated.  After I activated the Publishing Infrastructure, the errors went away.

However, ultimately the real solution was to remove the publishing features from the Master Page since I wasn’t using them anyway.

This problem came about when we re-used the master page from our main intranet on users’ My Sites.  Since the My Site doesn’t use the publishing features, every time someone hit a my site, these errors would show up in the event log.

Setting up automated backups for ESXi can be a little tricky. After quite a bit of searching I finally came up with these steps that worked for me. I don’t take credit for any of these. I’m just providing you with the steps that I followed and the posts that helped me.

DISCLAIMER: There are some unsupported steps here. Proceed at your own risk!

Step 1: Enable ssh access on your ESXi host. To do that, follow these steps (originally from this post).

NOTE: You actually don’t have to enable ssh for this, I just find it convenient to have remote access to the host machine.  If you don’t want to enable ssh, perform steps 1-3 below, then skip on to section 2.  When you reach section 3, rather than using ssh, you’ll just execute the commands directly on the host machine at the console.

1. From the console on your host (actually on the host machine – not using the Remote Client), press ALT-F1
2. Type in the text “unsupported” and press ENTER. Note: the text will not appear as you type.
3. Enter the root password
4. Enter the command

   vi /etc/inetd.conf

5. Find the line starting with “#ssh”. You can search by pressing / and then typing ssh and pressing Enter.
6. Once you’ve found that line, place the cursor on the # and press x. This should remove the # character.
7. Now type

   :wq!

   and press ENTER to save your changes.

8. Next you must restart the inetd process. To do that, you first need the process id. At the console, enter

   ps | grep inetd

   You should get something that looks like this:

   1273 1273 busybox      inetd

   In this case, 1273 is the process ID.  To kill the process enter kill -HUP <process id>.  So in this case, you would enter the following command:

   kill -HUP 1273

9. SSH is now enabled on your ESXi host machine. To test this you’ll need an ssh client. I’m using a mac so I can open terminal and enter the following command:

   ssh root@<myhost>

   If you’re using windows you may need something like PuTTY or Cygwin.

Step 2: Configure your backup datasource

Next you need to figure out where your backups will be stored.  It can be on the ESXi host if you like, but really it should be on a SAN or NAS or something like that.  I’m using a Netgear ReadyNAS NV+ and it works great.

1. Create a share where your backups will be stored and call it something like “backups”.  Make sure that anyone can read/write to this share.
2. Open the VMWare Infrastructure Client and connect to your host machine as root.
3. Go to the Configuration tab for the host and click Storage.
4. Click Add Storage… and create a new datastore pointing to the share you created in step 1.  Call this datastore something descriptive…  like “backups”

Step 3: Set up your backup script.

I found this great backup script.  It lets you specify the VMs you want to backup in a separate text file which makes it really easy to add new VMs to the rotation.  It also lets you keep a rotating number of VMs if you don’t want to over-write each backup daily.

1. Download the backup script.
2. Copy it to the same backup share you created in the last section.  You can create a directory called _scripts to keep it separate from the actual backups.
3. Open the script and customize the variables at the top.  In my case, all I had to modify was the VM_BACKUP_VOLUME variable.  If you named your datastore “backups” you can edit this line to look like this:

   VM_BACKUP_VOLUME=/vmfs/volumes/backups

    

4. Create a new text file in the same directory as the backup script called VMs.txt.  In this file enter the name of each VM you want to back up.  Each VM name should be on a new line.  Your file should look something like this:

   VM1
   VM2
   A_THIRD_VM

5. Connect to your host using ssh i.e. ssh root@<myhost>
6. Enter this command:

   vi /var/spool/cron/crontabs/root

7. Move the cursor to the very end of the file. Press a. This will enter edit mode.
8. Press Enter to create a new line and then add the following text:

   02 02 * * *  sh /vmfs/volumes/backups/_scripts/ghettoVCB.sh /vmfs/volumes/backups/_scripts/VMs.txt

   This will run the backup script at 2:02 AM every day. You can change the frequency or time of day if you like. This crontab reference is helpful.

9. At this point your entire file should look something like this:

    #syntax : minute hour day month dayofweek command
   01 01 * * * /sbin/tmpwatch.sh
   01 * * * * /sbin/auto-backup.sh #first minute of every hour (run every hour)
   00,10,20,30,40,50 * * * * /sbin/decodeSel.sh #Every 10 minutes, translate the latest IPMI SEL data
   02 02 * * *  sh /vmfs/volumes/backups/_scripts/ghettoVCB.sh /vmfs/volumes/backups/_scripts/VMs.txt
   

   If you are satisfied, press Escape.

10. Type the following command to save the file and exit::wq!
11. The backups are now scheduled

At this point, you now have backups scheduled to run every day at 2:02 AM. You can change which VMs are backed-up by editing the VMs.txt file you created. You could even set up multiple cron jobs with different backup frequencies, and different VMs.txt files to back up different VMs at different times.

Before finishing however, you might want to do a test run. If you are still connected to the host machine using ssh, enter the following command:

sh /vmfs/volumes/backups/_scripts/ghettoVCB.sh /vmfs/volumes/backups/_scripts/VMs.txt

IMPORTANT: This will take a while if you have large VMs. You may want to edit VMs.txt to target a single small VM as a test run. If it works for that, it should work for the rest.

One last thing. Unfortunately, the crontabs/root file isn’t saved after a reboot. In order to avoid adding the scheduled job after each reboot you may have to follow one of the techniques described here.