SharePoint: SQL Server Design Tips

SQL Server databases are the largest consumers of disk space in SharePoint. Thus, designing your disk system for maximum performance at the database level is more crucial than for any other level. Here are some general design tips.

Don't virtualize SQL Server. If possible, don't virtualize SQL Server because it's already an integration platform. If you have to virtualize SQL Server, try to limit the virtualization to test or development systems. Avoid using it for production systems because it raises the bar in disk engineering to get good performance. SQL Server databases stored in virtualized disk files are inherently slow compared with dedicated physical disks.

Use multiple logical drive letters. It's usually a good idea to break up SQL Server databases into multiple logical drive letters because database files, transaction logs, backup files, and temporary databases (tempdb) benefit from having multiple independent sets of disk spindles. Adding more spindles spreads the load across multiple parallel operations when data is being written to the database.

If you're going to use a SAN or virtualized environment, make sure you understand where those logical drive letters will be mapped. For example, if the D, E, and F drives will all point back to separate LUNs on your SAN, but those LUNs will be part of the same storage group and same set of physical disks, splitting those files into multiple drive letters will add complexity without significant performance gains.

Use RAID 10 judiciously. RAID 10 is great, but you might not be able to justify it for all applications. For example, it might be overkill for backup files. Balancing disk performance and cost is a reasonable trade-off. One possible design is to use:

• RAID 1 on boot disks
• RAID 5 on data disks
• RAID 10 on log disks
• No RAID or RAID 5 on backup disks

Break large content databases into multiple database files. If you have large content databases, you can engineer better performance by breaking each large database into multiple database files. Each database file should be on a separate disk.

Presize SQL Server databases. SQL Server databases can be set to automatically grow as needed, but this can lead to massive file fragmentation. Presizing the databases to a sufficient size at the outset helps ensure contiguous file allocations. Note that SQL Server's tempdb database is heavily used by SharePoint, so you should presize it to about 20 percent of the size of the single largest content database.

If you want to use automatic database growth settings instead of presizing your databases, you should set the databases to grow in 50MB to 100MB clumps and not by percentage. Setting a 100GB database to grow in 10 percent increments means the database essentially stops to add 10GB or more on each increment. Using a small clump size will lead to more frequent, but smoother, steady state (i.e., continuous) growth.


Design a High Performance Database Environment


If you use RAID and follow the general design tips, you can design a high-performance database environment that's also highly available. This is essential to a smoothly running SharePoint system.

Top 10 SharePoint 2010 Configuration Mistakes

1: Scrimping on SharePoint's RAM or Hard Disk Space

A poor, defenseless SharePoint server working as hard as it can to keep users happy, but having its hands tied because of limited resources. This situation is usually a casualty of aggressive virtualization. Virtualization itself isn't bad, but it must be done intelligently and without sacrificing SharePoint's ability to do its job.

If SharePoint finds itself starved for RAM, it starts shutting off functionality so that it can fit into the available space. It also caches less in the web application pools and recycles those pools more often. Less caching and more recycles result in a degraded end-user experience, as SharePoint must compile the same ASP.NET code over and over. And no one likes unhappy users, not even their mothers.
The solution to this particular issue is easy: Add RAM. Microsoft has published the hardware requirements for SharePoint 2010 in the TechNet article "Hardware and software requirements (SharePoint Server 2010)."  These requirements state that at the very least, each SharePoint 2010 production server should have 8GB of RAM and a C drive with at least 80GB. In many cases, that won't be enough. If your servers are in production, you can watch their memory utilization to see whether they use the entire 8GB of RAM. If so, they could use more. If your servers are not yet in production, you can use a variety of load-testing tools to simulate your intended load and see how the servers hold up. For example, you can download the Microsoft Load Testing Kit, part of the SharePoint Administration Toolkit.

As for your C drive, SharePoint itself doesn't need much space, but Windows does. After all, your server has several years of Windows patches to look forward to. While you're adding drive space to your machine, consider adding a secondary drive as well. This drive is a great place to store all the files that you use when you install SharePoint. All the third-party installation files can go there too. You can also have SharePoint put its log and Search index files on this drive. This approach takes some pressure off the C drive. Happy C drive and happy end users equal a happy SharePoint server administrator.

2: Using Virtualized Microsoft SQL Server

Virtualization isn't bad. But virtualization allows administrators to make mistakes on a much grander scale. Take virtualizing SQL Server. In the context of SharePoint, this process can be especially painful. The main mistake I see when virtualizing SQL Server is overcommitting the host, be it through RAM, CPU, or drive space. Because everything in SharePoint is stored in SQL Server, if SQL Server is slow, SharePoint is slow.

The obvious solution is to move SQL Server to a physical box, on which it doesn't need to share resources. Moving SharePoint's SQL Server instance is easy, thanks to aliases. This process outlined with pictures, at www.toddklindt.com/sqlalias.

If you can't get a physical SQL Server box, then at the very least ensure that your virtualized SQL Server instance has a fighting chance. First, make sure that its virtual drives aren't thin provisioned. I/O is one of the areas in which virtualized SQL Server struggles the most, and thin-provisioned drives exacerbate that problem. Also try to put the SQL Server guests' virtual drives on their own spindles on the host. Doing so should improve I/O by preventing SQL Server from fighting other guests for time with the drives. Finally, you shouldn't allow the virtualization host to overcommit its RAM. If the host must swap to meet its RAM obligations, then it's slowing down SQL Server.

Brent Ozar has recorded a brilliant video on how best to virtualize SQL. Go get some wine and pizza, invite your fellow SharePoint admins, dim the lights, and watch that video. You'll learn a lot.

3: Using the Farm Configuration Wizard

 

Using the Farm Configuration Wizard was a pretty common mistake when SharePoint 2010 first came out but thankfully has diminished as our familiarization with SharePoint 2010 has increased. The wizard's list of atrocities is long, so I'll just cover some of the better known ones. First, and maybe most heinous, is that all the databases that the wizard creates have nasty globally unique identifiers (GUIDs) at the end of their names. The wizard also creates a content web app, at http://servername, that just doesn't scale well. To add insult to injury, the wizard creates your My Site host on that same web app, at http://servername/my. Finally, the wizard encourages you to create service applications that you might not actually use. It's tough to resist the siren song of those check boxes, I know. 


The Farm Configuration wizard leaves its dirty handprints all over SharePoint, and it can be a challenge to clean up all of them. However, a few places can be easily fixed. Start with your web apps. Create a web app for My Site and give it a Fully Qualified Domain Name (FQDN), such as mysites.company.com. Create a My Site host at the web app's root. Use the Windows PowerShell cmdlet Move-SPSite to move any My Site to one content database, and then attach that content database to your new web app. You'll also need to adjust your User Profile Service and tell it about your new My Site location.

Next, clean up your service applications. Go through your list of service applications and delete any that you aren't using. You gain no benefit from having a service application that you aren't going to use for another six months. After you've deleted unnecessary service applications, stop the associated service instances (also called services on server) that power them. If possible, remove the GUIDs from the service application database names. The technique for completing these tasks varies among service application; the Microsoft article "Rename or Move Service Application Databases (SharePoint Server 2010)"  has directions for all the service applications. Of course, take good backups before doing any of this.

4: Using an Incorrect URL when Creating a Content Web App

 

Like any relationship, SharePoint and Microsoft IIS have communication problems from time to time. Web app creation is one of those times. SharePoint doesn't tell IIS about changes that you might make to a web app after it is created. For instance, if you create an Alternate Access Mapping (AAM) for a web app in Central Administration, you still need to go into IIS and add the host header for the new address.

The issue is compounded when SharePoint farms that you never thought would need to be accessible from the Internet suddenly need to be accessible from the Internet. Budding SharePoint administrators commonly give their web apps short URLs, such as http://portal, to save users some typing. Of course, that URL doesn't route across the Internet, so the web app needs a fully qualified URL added to its stable of AAMs. Not only is this new URL not written to the IIS host headers, but it's also missing from all the alerts, workflows, and anything else that saves URLs -- all those items have the old URL hard-coded in. Because SharePoint didn't write any additional URLs to IIS when they were created, it won't write them to any new SharePoint servers that are added to the farm. Nor will SharePoint write these changes to IIS if the Microsoft SharePoint Foundation Web Application service instance is stopped and started.

This issue might not seem like a big deal, but it has bitten many people at the worst possible time: during an outage. In a few cases, administrators have lost or needed to rebuild a SharePoint server and forgotten about the host headers that they put in manually months earlier. SharePoint is up and going, but when browsing to SharePoint, end users get the blue IIS 7 splash page instead of the SharePoint page that they were expecting. Again, unhappy users usually mean unhappy administrators.

Because SharePoint writes host headers only when a web app is created, you can't fix problematic web apps. You'll need to recreate them. That's good news and bad news. The good news is that you won't lose any of the content that your users worked so hard to create. The bad news is that you will lose all the settings that you worked so hard to create.

The first step is to make notes of all your web app settings. In most cases, there won't be many, and you'll be familiar with any changes that you made. Then, detach the content databases from your web app. Keep them safe; you're going to need them. Next, make a copy of the web.config file for that web application. Some settings, such as forms-based authentication (FBA) and BLOB cache settings, are in that file. Finally, go into Central Administration and delete the web app. Tell SharePoint to delete the extra stuff. The scary part is over.

Now, recreate the web app, but do it right this time. First, enter the correct, fully qualified URL in the Host Header box. Do your end users a favor, and put the web app on port 80, as Figure 1 shows. Under the Security Configuration settings, accept all the defaults, even if you're going to use Kerberos or SSL. You can change those settings later, and you want to make sure that the web app works correctly before you apply fancy security settings. Doing so helps in any troubleshooting that you might need to do. Under the Application Pool settings, pick an existing application pool.

It is important to give your content databases distinct names. You should be able to look at a content database name and know exactly which web app that database goes with. This is another one of those things that doesn't usually seem important but is priceless in a disaster-recovery situation. If the content databases that you detached from the web app before you deleted it didn't have such names, then take this opportunity to right that wrong when you recreate the web app. Give the new content database a good name, then use the PowerShell cmdlet Move-SPSite to move the site collections to that new database. If your content database already has a good name, enter it during the creation of the new web app. If you had multiple content databases, attach the rest after the web app is created.
After the web app is created, you can tweak settings as needed. Most settings can be changed in Central Administration. If you made any changes to the web.config file of the original web app, now is the time to copy those changes to the newly created web.config file. You can use a program such as Notepad++  to compare the two files. You should now have a well-created web application that you can trust in times of crisis. 

5: Running Web Apps or Service Apps in Separate App Pools 

Web apps and service applications run inside of an application pool, which is a W3WP.exe process that runs on your server. Unless you have reason to do otherwise, you should run all SharePoint web apps inside one application pool; the same goes for the service applications. Running each web app in its own application pool makes inefficient use of the server's memory. Each application pool has a minimum overhead of more than 100MB, and its memory footprint increases as it caches content that's rendered frequently. Figure 2 shows multiple W3WP.exe processes running as sp_webapps, the result of web apps running in separate application pools. We've all experienced SharePoint slowing first thing in the morning because the app pools recycle overnight and need to warm up and cache that content again. Well, multiple application pools mean that the same content is cached multiple times. Most users are impatient. I'm sure that studies would show that they spend the time waiting for SharePoint to respond by thinking of ways to punish us for SharePoint's poor performance. 

For service applications, this problem is easy to fix. First, make sure that you have a good service application pool to use. I recommend calling this pool Default SharePoint Service App Pool so that it floats to the top of all your drop-down lists. Use a dedicated sp_serviceapps account for the pool's identity. Most service applications allow you to assign them to a new service application pool by modifying their properties in Central Administration. If the option is unavailable there, look for it in PowerShell.

Web applications are a tougher matter. There's no easy, out-of-the-box way to change the application pool that a web app is using. Fortunately, we have PowerShell at our disposal.

6: Using One Account for Everything 

 

Security is complicated, and SharePoint doesn't necessarily make it any easier. Using just one account -- maybe even the coveted Domain Administrator account -- is so easy. We've all done it, even though it's a bad idea. When you use an existing account, you open up SharePoint to several security issues. Anyone who knows the account password can do anything in SharePoint, so you can't separate duties. You also lose the ability to audit who made which changes. And if that common account password is compromised or needs to be changed, you jeopardize SharePoint's uptime as well. Even if you use one dedicated account for SharePoint, you leave yourself vulnerable to attack. If that account is compromised via a security exploit, the bad guys will have access to everything in SharePoint.

To fix this mistake, start by creating the accounts. Add the sp_webapps and sp_serviceapps accounts as managed accounts. Use the techniques that describe in Mistake  No 5 to fix your web app and service application accounts. You can change the default content access account for the Search service application at the Search Service Application page. Under Central Administration, Security, Configure Service Accounts, you can change the accounts that other processes use as well. (You can even change the Farm Account there. I've done so in test environments but haven't been brave enough to do it in production.) If you're using the User Profile Service, make sure that your new sp_userprofile account has the correct permissions in Active Directory (AD), and recreate your AD connection in the User Profile Service.

7: Keeping Default SharePoint Database Settings

 

When SharePoint creates its multitudes of databases, it makes some bad assumptions. Take the autogrow settings: The database files grow by 1MB at a chunk, almost ensuring that they're going to autogrow with every upload. Not only does this slow down SQL Server (which slows down SharePoint), but it also results in database files that are spread all over your drives in itty-bitty 1MB chunks.

SharePoint also creates most of its databases, notably the Config and Content databases, with the recovery model set to Full. Although this is great if you want to recover data, you must manage the process correctly or those sneaky .ldf files will slowly, methodically fill your hard disk. If you think users get upset when SharePoint is slow because of fragmented databases, you should see how angry they get when SharePoint stops completely because the SQL Server drives are full.

To fix this mistake, set your databases' autogrow settings in such a way that they don't need to grow frequently. For most farms, I recommend changing the 1MB autogrow to something like 500MB or 1GB. Autogrow should also be a last resort. Someone, either the SharePoint administrator or a dedicated DBA, should pregrow your databases so that autogrow is unnecessary.

Your recovery model setting needs to be consistent with your disaster recovery plans. If you need your transaction logs, make sure you're performing routine log backups to keep those .ldf files in check. If you don't need your transaction logs, then consider switching your databases to the simple recovery model. Doing so will keep your .ldf files from swelling up like a nasty bee sting.

8: Not Enabling BLOB Caching

 

We all want SharePoint to get files to the users as quickly as possible. However, more often than not, I see SharePoint farms without BLOB caching enabled. BLOB caching is one of the easiest and least expensive ways to improve SharePoint performance. Not only does it help to get files to users more quickly, but it also eases the burden on SQL Server. Everybody wins.

This might be the easiest solution so far: Enable BLOB caching, of course! BLOB caching is actually a function of IIS; SharePoint just takes advantage of it. Therefore, to enable BLOB caching requires a change to each web app's web.config file on each server. Fortunately, the setting already exists and just needs to be enabled. By default, the web.config files are in a directory under C:\inetpub\wwwroot\wss\virtualdirectories. Each web app has a directory and a web.config file. Open one of these files and look for the following line:


<BlobCache location="C:\blobcache\14" path="\.(gif|jpg|jpeg|jpe|jfif|
bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|
mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />

To enable BLOB caching, replace "false" with "true" and save the web.config file. You should also move the file to a directory on a drive other than the C drive. The maxSize parameter is measured in gigabytes, with a default of 10GB. If the space is available, you might want to increase this size.
If editing this file in Notepad on all your servers isn't your idea of fun, you can use PowerShell to automate the process. You still need to perform the process on each server, but using PowerShell is quicker and reduces the chances of a mistake.

9: Not Installing a PDF iFilter 

 

Most organizations have a tremendous number of PDF files in their SharePoint farms, and those files represent a wealth of information. End users want to be able to discover that information by using SharePoint Search. Getting users excited about SharePoint Search is a great way to get them excited about SharePoint in general.
Installing a PDF iFilter is fairly easy. Adobe has a free PDF iFilter that you can install. You can find the download link and detailed installation instructions in the Microsoft article "SharePoint 2010 - Configuring Adobe PDF iFilter 9 for 64-bit platforms." You need to install the iFilter only on those SharePoint servers that run the Search Index role, although installing it on the rest of your SharePoint servers doesn't hurt. If you have a large farm and want to reduce the time needed to index your PDF files, you can use the PDF iFilter from Foxit. This product has better performance than the Adobe iFilter but isn't free.

10: Not Pointing Your SharePoint Servers at Themselves

 

When SharePoint works, it is magnificent. When it doesn't work, it can be a nightmare to fix. For this reason, anything you can do to ease troubleshooting is time well spent. To that end, I make sure that every server in the SharePoint farm points to itself for all web apps. If I get sporadic reports about SharePoint not responding, I can easily use RDP to log in to each server and try to pull up SharePoint. If this attempt works, then I know that the server is working. If SharePoint does not come up, then I know in exactly which Microsoft User Location Server (ULS) logs to look for the relevant errors. No worrying about which web front end the load balancer sent my request to. The quicker you get to the correct log files, the quicker the problem is resolved.
Pointing your Search indexer at itself has another advantage: It improves performance for your end users. If you don't point your Search server at itself, then when it starts to perform a crawl, it lets DNS do its work and then starts crawling whichever web front end DNS points it to. That server is most likely the same one that is sending pages to your end users. Making the server do double-duty means that everyone waits longer. Pointing the Search server at itself means that your web front end doesn't need to handle that traffic and can get back to doing its #1 job: keeping users happy.
There is a simple fix for this mistake: Open the hosts file (C:\windows\system32\drivers\etc\hosts) on each SharePoint box, and add all the URLs that SharePoint knows about. Point those URLs to 127.0.0.1, which translates to "this computer." Figure 3 shows how this file looks in a typical SharePoint environment.

 

 

 

 

SharePoint 2010 Service Account

Account name	Role	Domain rights	Local SharePoint Server rights needed	SQL rights needed
sp_install	Used to install SharePoint binaries.	Domain User	Local administrator on all SharePoint boxes	dbcreator and securityadmin SQL roles
sp_farm	Farm account. Used for Windows Timer Service, Central Admin and User Profile service	Domain User	Local Admin during UPS provisioning, log on locally right	None
sp_webapp	App pool id for content web apps	Domain User	None	None
sp_serviceapps	Service app pool id	Domain User	None	None, unless using Office Web Apps. Them must give access to content databases manually
sp_search	Search process id	Domain User	None	None
sp_content	Account used to crawl content	Domain User	None	None
sp_userprofile	Account used by the User Profile services to access Active Directory	Must have Replicating Change permissions to AD. Must be given in BOTH ADUC and ADSIEDIT. If domain is Windows 2003 or early, must also be a member of the "Pre-Windows 2000" built-in group.	None	None
sp_superuser	Cache account	Domain User	Web application Policy Full Control Web application super account setting	None
sp_superreader	Cache account	Domain User	Web application Policy Full read Web application super reader account setting	None

See http://technet.microsoft.com/en-us/library/ee721049.aspx 

Again, these are just recommendations. You may end up using more accounts if you have multiple application pools, for instance. Your particular farm may require different accounts.

SharePoint Business Architecture

Architecture refers to the art of building. The word "architecture" has many meanings. Probably, the most understood meaning is "the art of constructing structures such as homes and buildings." The architect designs the blueprints of the home or building, taking into account factors such as design, space, light, materials, stability, load, and future needs.

Architecture is important because it accounts for the functional and nonfunctional requirements early on. Microsoft Office SharePoint Products and Technologies are powerful tools that increase collaboration and sharing of content. If implemented correctly, SharePoint can store and serve a vast quantity of information very well. However, without proper architecture and governance, a SharePoint deployment can become an unorganized collection of sites, links, users, and documents that hampers productivity and makes it harder to find information.

A good architecture plan and governance plan  lay down guidelines for deploying SharePoint as a solution to common business challenges. The architecture of SharePoint includes designing and allocating the hardware infrastructure needed to support the site, listing out the sites and site hierarchies that will serve the needs of the business, establishing users and roles that will be given permissions to the sites, establishing the relationships between sites, and planning for the needed site features, site customizations, and site and list relationships (which include how data will be rolled up and aggregated from sites and lists to provide an overview of information).

A good governance plan outlines the administration, maintenance, and support of the SharePoint environment. The governance strategy seeks to ensure that SharePoint is used in accordance with the designed goal and that best practices are followed to keep the portal manageable and usable. Best practices include processes for operation in the portal for tasks such as creating sites and lists, assigning permissions to users, using consistent naming conventions, and generally enforcing the guidelines.

When creating a building, architectural concerns include data gathering, planning, and the design of that building. The SharePoint architect must design the SharePoint building to withstand the test of time (meaning the architect must future-proof the implementation by building in robustness and resiliency) and, based on future client requirements, be able to expand easily (in other words, scalability with an eye on future upgradeability of SharePoint installation based on factors such as business need, hardware, software resources, and so on).

For SharePoint, there are three levels of architecture: hardware architecture, software architecture, and information architecture.

Hardware Architecture

To deliver a robust SharePoint 2010 environment, it is necessary to carry out technical design, which looks at all areas of SharePoint 2010 concerning the equipment it will run on or be connected to and systems and processes it will interface with. The following is a list of planning requirements:

• System requirements determining what is required to deploy SharePoint 2010.
• Services architecture determine what service applications are defined and how are they structured.  
• Logical architecture presents the design in terms of isolation. This planning task looks at farms, service applications, Web applications, content databases, site collections, sites, zones, MySites, and so on.
• Authentication examines authentication methods, such as claims-based authentication topologies.
• Server hardening this task focuses on server snapshots, ports, protocols, and the Web Server, Application Server, and Database Server roles.
• Business continuity examines the business decisions, processes, and tools put in place to handle a crisis. A crisis can affect the organization or be part of a local, regional, or national event. Business continuity and disaster recovery are huge areas in SharePoint and planning for them is an important part of ensuring a resilient and robust platform.
• Performance and capacity determines the process of mapping the design for SharePoint 2010 to a farm size and the hardware needed to support the business goals. 
• Virtualization SharePoint 2010 is fully supported when deployed in a Windows Server 2008 Hyper-V environment. This task examines the licensing and topology.

Software Architecture

The software architecture of SharePoint is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them. So decisions to be made include determining what components of SharePoint are needed, what will be visible, and the structure of SharePoint. For example, is SharePoint going to be treated as an out-of-the-box solution, slightly modified with internal applications, or will it include third-party additions? Will it simply need just team site components (for example, the free SharePoint Foundation version), or do you need more service application, enterprise content management, or metadata features, such as those provided through SharePoint 2010 Enterprise?

Software architecture examines SharePoint from a site and solution planning perspective, taking into consideration site components, security, governance, enterprise content management, Web content management, managed metadata, business intelligence, data and processes, access services, quota management, and social computing.

As an example, suppose that you’re going to implement SharePoint in an organization that already has SharePoint but needs to expand. They have a third-party tool providing some functionality that the client finds useful. From scoping the information architecture, you found how much usage it gets, how the data is used, how it flows, and so on. From further investigation of the software architecture, you find that the relevant tool cannot grow with the service. This means revisiting the functionality in terms of the information architecture and finding an alternative, which then drives the software architecture.


Information Architecture

Information architecture involves studying the type and amount of information used within an organization, organizational structure, information flow, process flow, and more. This is an extremely important aspect of the Plan phase. Without it, SharePoint is not defined to meet the client requirements, because information architecture leads to SharePoint user strategy in terms of content management. Identifying the organizational information and management information goals combines the work of information analysts and business analysts, coordinated by the project Manager and feeding back to the SharePoint architect.

Large organizations have documentation plans and methods of managing their data across the organization (for example, retention plans and archive plans), and some use information analysts to manage, coordinate, and categorize how members in the organization deal with information. Additionally, organization face legal and regulatory compliance requirements that directly influence how data is retained long term. In the United States, for example, the Sarbanes-Oxley (SOX) Act established record-retention rules in July 2002. It is highly recommended any company have a records-retention policy that complies with regional and national laws.

Another benefit of a good records-retention policy is a decrease in storage costs.The information analyst details from the ground level the organizational data concerning information standards and policies set out by the business. Information architecture establishes information control and compliance policies so that accumulating information is done in a well-managed way and does not create data chaos.

SharePoint 2010 provides enterprise content management tools that can help lower costs associated with the control and storage of information, decrease complexity, and increase user participation relating to content control. Combining SharePoint 2010 with Office 2010 takes information management to a higher level by extending information control from the desktop environment to SharePoint 2010 sites and content.

The aim of information architecture in SharePoint is to reduce the manual end user actions related to metadata, to scale policies and processes across all types of content in an organization, and to increase compliance and transparency. To meet this goal, there must be an examination leading to the creation of an organizational taxonomy. During the Design phase of the SharePoint project, the information analyst (working with the SharePoint architect) creates a taxonomy for the organization by examining metadata and information policies.

The business analyst can provide, through the collection of user requirements, an understanding of what the typical content life cycle is in the business. This shows how end user content becomes managed content. Typically, managed content begins its life as temporary information created by the individuals in the organization, leading to work in progress (and this means multiple individuals working on the same content) and in the backdrop of retention and disposition (business teams or individuals deciding on whether documents should be archived and what their state is, either approved, published, or other). SharePoint 2010 provides tools to ensure the content life cycle can be designed and adhered to. Enterprise content types, document sets, information management policies, metadata, term sets, and content organizers can be established using SharePoint 2010 document management features. 

Here are some basic procedures for setting the information architecture for SharePoint 2010:

• Carry out an investigation and inventory of existing content.
• Classify the content by performing the following tasks: Look for definitions of structure, policy, and defaults; Identify organizational-level content by enterprise, department, and team; Define what “general use” content is.
• Organize content into enterprise content types and document sets, keeping the following factors in mind: Content types are where there are definitions of structure, policy, and defaults; Content types can inherit from other content types; Document sets are where the work spans multiple documents.
• Decide where information management policies apply. When doing this, be sure to consider access permissions, auditing, user restrictions (for example, no printing), retention, and deletion.
• Decide on applicable metadata by performing the following tasks: Define customized columns, and associate them with documents and lists; Define any cases where the system or user might take different actions based on the characteristics of an item. Note that the characteristics of the item are metadata; Find out what common things users will want to sort or filter items on; Find out what words or phrases users are likely to tag items with; Use Choice or Lookup columns in SharePoint 2010 sites; Use the existing taxonomy if the organization has one.
• Map the physical flow of the document, including the sites, lists, and libraries where the content will be physically located throughout the document’s life cycle.

 

SharePoint Governance Architecture

Governance is the cornerstone to any successful SharePoint implementation. Without it, you are doomed to fail eventually—a lesson that many organizations learn the hard way.

Because governance matters so much, there is a lot of information about it, starting with the TechNet SharePoint Server 2010 Governance Resource Center.

Many of these resources focus on how to create a governance framework: how to assemble the people, policies, and procedures and how to develop a governance plan. But all too often, such discussions leave out a fourth equally important component of governance: technology.

You must understand the technology that you are trying to govern; you can’t ask it to do something that it cannot do. And you should use the technology to facilitate governance. In this first of a series of articles, I will explore the technical side of governance and thereby answer several vital questions:

• What does a governable SharePoint implementation actually look like?
• What is the physical and logical architecture of such an implementation?
• How many farms, servers, web applications, content databases, site collections, and sites does such an implementation have?

The initial answer to all three questions is, unfortunately, “It depends.” The answer depends not on SharePoint, but on what you are trying to achieve by using SharePoint. The best way to get to the answer that applies to you is to follow my four-step Architecting Governance process. By following this process, not only can you answer the previous question, but you will have a logical, physical, and governable architecture that meets your business requirements.

Given the limitations of space and time, this article will focus on the process itself, which involves four major steps. In future articles, I will explain how you can apply the process to specific scenarios, to gain prescriptive guidance towards successful architectures. And I will show you how to use the technology to automate and enforce your governance policies.

Step 1: Define Your Requirements

 

Step 1 of Architecting Governance is all about requirements. As a consultant, you spend probably 80 percent of your time helping customers to define requirements and develop discipline around requirements gathering. After all, you must understand the requirements for any solution before you can effectively design that solution. SharePoint is no different. 

The problem that I observe is that SharePoint implementations involve a lot of requirements. So I find it helpful to categorize them, and then identify which categories are salient at each step in the Architecting Governance process. I suggest that you group your requirements into these categories:

• Business requirements—These are the requirements that really matter. They define the business purpose of the solution that the customer is asking you to create. Whenever possible, avoid polluting business requirements with technical requirements. More often than not, technical requirements are artificial. When a business customer says, “I need a sub site that does x,” or “I need a button that does y,” that customer is casting themselves as a technologist—a solutions developer. Encourage them to take a step back and describe the desired result (x or y) without mentioning technology. Let the technical solution be developed by those who know the technology.
• Technical requirements—Occasionally, technical requirements must be considered. For example, if your mobile sales force is going to use their iPads to access the solution that you are providing, then the ability to access the solution from iPads is a valid technical requirement. Such technical requirements more often relate to architecture—interoperability with other solutions—or infrastructure than to functionality or usability.
• Project requirements—These requirements relate to the creation of the solution, not to the business purpose of the solution. Budget and deadlines are prime examples of project requirements.
• Information-architecture requirements—Information architecture, in its most traditional definition, relates to how content is described, organized, and discovered.
• Information-management requirements—Information-management requirements define how the content is managed over its lifecycle: how is it created, maintained, and archived or deleted. These requirements relate to security, records management, auditing, and compliance.
• Service-management requirements—Behind the solution, the content, and the information is the service itself. Service-management requirements describe IT assurance expectations: recovery, availability, and performance. These requirements lead to service level objectives (SLOs) or service level agreements (SLAs).

Categorizing requirements is valuable for several reasons. First, you can identify the dependencies between requirements. This ability allows you to proceed through the requirement-gathering process in a logical manner. 

Building an Offline Learning Kit for SharePoint 2010

SharePoint 2010 product and documentation teams and, the developer evangelists have done a great job of providing developers and IT Pros with learning material online. This content can be accessed mainly through the product homepages on MSDN and TechNet websites. Channel 9 also has lot of content.

Besides the articles, white papers and Visio diagrams, there is lot of good content in Video format. Accessing this content requires Internet connectivity and the high-quality videos require a lot of bandwidth. This problem gets aggravated if you a large developer team or if you travel a lot.

What’s the solution? Based on my experience, the best (and sometime only) way out is to download all this great content and have it available offline on your laptop, on file server or on your team’s SharePoint site. The flip side is the bandwidth required and time (and patience) it takes to download.

How to Track the Content?

First problem is keeping track of the content being published. I keep track by following blogs and links given below:

• Newly published content for SharePoint Server 2010 on TechNet. RSS feed
• Recently Published Content on SharePoint Developer Center. RSS Feed
• GetThePoint blog
• Microsoft SharePoint Developer Documentation Team blog
• To the SharePoint blog
• Channel 9 – SharePoint 2010 and SharePoint tags
• Microsoft Download Center

Let me know if you know of a better URL to keep track of all the content being published.

Downloading the Content

In this section, I have listed the various sources I found for downloading content. At some places, multiple video formats are available.

White Papers and Guides:

• SharePoint 2010 Resources for IT Professionals
• SharePoint 2010 Resources for Developers 
• Downloadable content for SharePoint Server 2010
• Microsoft Download Center

Presentations:

• SharePoint Server 2010: Advanced Developer Training Presentations - Collection of in-depth training presentations for SharePoint Server developers
• SharePoint Server 2010 Advanced IT Pro Training presentations - Collection of in-depth training presentations for SharePoint Server IT pros

Online Trainings:

MSDN - Learn SharePoint Development

• Get Started Developing on SharePoint 2010
• SharePoint Advanced Developer Training
• SharePoint Enterprise Search Developer Training

TechNet - Learn SharePoint Server 2010

• Get started with Microsoft SharePoint 2010
• Advanced training for SharePoint 2010
• SharePoint 2010 Enterprise Search IT professional training
• SharePoint for Internet Sites Implementers' Course
• SharePoint Enterprise Content Management Implementers' Course
• SharePoint Server 2010 Business Composites Training
• SharePoint Server 2010 Business Intelligence Insights Training

Channel 9 Videos:

• SharePoint Channel
• SharePoint 2010 Channel
• Tech·Ed North America 2011 

Offline Training Kits:

• SharePoint 2010 Developer Training Course
• SharePoint and Silverlight Training Course
• Office 2010 Developer Training Course
• SharePoint 2010 and Windows Azure Training Course
• SharePoint 2010 and Windows Phone 7 Training Course
• Silverlight 4 Training Course

More offline training kits are available here.

How do I – Videos

• How Do I Videos for Office Developers
• SharePoint Server 2010 Help and How-to Videos

Other Kits

• Developing Applications for SharePoint 2010
• SharePoint 2010 Reference: Software Development Kit
• Microsoft SharePoint 2010 Administration Toolkit v2.0

Office 365 content

• Office 365 Jump Start
•  Office 365 Developer Training Course
• Microsoft Office 365: Connect and Collaborate Virtually Anywhere, Anytime

Once you are through with all the downloads you would have exceeded 40Gb if you selected high quality video option (wherever available) !!

sharepoint 2010 branding

This page contains a list of resources useful for SharePoint 2010 Branding, User Interface, Design and Web Content Management solutions.

As I was researching SharePoint 2010 Design for our new book Beginning SharePoint Designer 2010 (wrox), I collected several links to good resources. I hope you find them as helpful as I did. I tried to bucket them in order of design/dev cycle. Why? Probably because I’m an Information Architect at heart. No reason really.

 

Key Resources

• The most important source of information is the SDK.
• The other critical SharePoint 2010 Branding & Customization resource is MSDN’s Real World Branding with SharePoint 2010 Publishing Sites

Planning and Design

Usability

• 20 Do’s and Don’ts of Effective Web Design
• Introduction to Good Usability
• 10 Useful Usability Findings and Guidelines
• Universal Usability Guidelines

Information Architecture

• Brief overview: “Information Architecture Deliverables”
• Information Architect by Peter Morville

Wireframes and prototypes

• Mockflow – I love love love this web app. WIreframes, interactive flow and site map all at once.
• Adobe Illustrator - I recommend Adobe Illustrator for the fastest wireframes with loads of content.
• Notable – I use this app religiously. Works so well for remote discussions about visual requirements.
• SharePoint 2010 Visio Stencils by John Chapman

Web Content Management

• WCM improvements in SharePoint 2010 – Mike Ferrara
• Content Strategy – Kristina Halvorson
• SharePoint 2010 Web page planning worksheet – Technet (What a gem for planning SharePoint 2010 publishing sites. Much better than SharePoint 2007’s page planner!
• Plan Pages in SharePoint -Technet
• IntranetFactory – Get a Content Inventory of your SharePoint site
• Jumpchart – a great tool to give to your marketing team to work on content while you are building your SharePoint solution.  It may sound hacky but it keeps marketing out of a site that isn’t yet finished. (that’s good for everyone)

User Interface Design

• SharePoint 2010 Layered Photoshop File! – Erik Swenson
• The Pursuit of Interface Design Simplicity -Luke Wroblewski
• Visible Narratives: Understanding Visual Organization -Luke Wroblewski
• Common Visual Design Misconceptions – Luke Wroblewski

Implementation

 

Microsoft SharePoint Designer 2010

Downloads:

• SharePoint Designer 2010 (32 bit)
• SharePoint Designer 2010 (64 bit)

CSS and HTML Resources

SharePoint 2010 requires a better understanding of CSS than you needed in SharePoint 2007. You really should be able to plan your css. If you can use the box model on purpose. If you know how to clear a float. If you understand a stacking index, you are okay. If you don’t, practice on plain old HTML first. Build your own HTML from scratch and style it. Draw a design and build it from scratch Plan it.

• XHTML™ 1.0 The Extensible HyperText Markup Language (Second Edition) -w3.org
• Grid Builder – developer.yahoo.com/yui
• CSS Box Model – css-tricks.com (Advanced CSS design patterns are used in SP2010. You need to understand the impact of changing a value in one place.)
• CSS Overflow (you need to know the impact of this property)
• CSS Overflow (css-tricks.com.)

CSS Sticky Footer – You may find yourself struggling to get your footer to stay at the bottom of your page. If you really want this to be a part of your site, test the layout and structure all alone to troubleshoot. Test with and without content on as many site templates as possible. CSS Sticky Footer, CSS Sticky Footer II, CSS Sticky Footer III

• JS Sticky Footer If all else fails, use this JS Sticky Footer. Be sure to test your site with real content!! If it’s not in the body of the site, look to the CSS overflow as the culprit.
• Understanding Floats (another advanced technique but helpful alternative to overflow)

Themes

• SharePoint 2010 Themes – Erik Swenson
• How the SharePoint 2010 Theming Engine Works – Buta No IE
• SharePoint 2010 keeps copy of modified theme

General SharePoint CSS and Markup

• SharePoint 2010 CSS Reference chart – Heather Solomon
• The Shiny New CSSRegistration Control in SharePoint 2010 – Buta No IE
• SharePoint 2010 CSS references in Master Pages – Erik Swenson
• https://www.nothingbutsharepoint.com/sites/eusp/Pages/Use-font-face-in-Rich-Text-Editor-of-SharePoint-2010.aspx

SharePoint Navigation

• Adding Navigate Up breadcrumb on your search center
• Very cool Mega Drop Down Navigation

Master Pages

Starter Master Pages

• Starter Master Pages on CodePlex – Randy Drisgill
• Starter-Master-2010-v1.0.txt – Kyle’s SharePoint Karate, theSUG.com
• Fixed Ribbon Starter Master Page – Kyle’s SharePoint Karate, theSUG.com
• Microsoft’s SharePoint 2010 Starter Master Page – MSDN
• GreenShade Free Master Page - TopSharePoint.com

Master Page Tips and Tricks that will save your ARSE

• Fixed Width Master Pages in SharePoint 2010 – Elumination Blog
• How to Create a Fixed-Width 3 Column layout – Tom Wilson, Styled Point
• Imtech Master Pages and Page Layouts Extension – Imtech
• Centered layout with CSS & v4.master on SharePoint 2010 – SharePoint Blues

Simply Cool Tips and Tricks

• Image slideshow with CQWP in SharePoint 2010 - Christian Stahl
• Sing Chan on general and SharePoint 2010 branding – Sing Chan
• Location aware content rollup – Serve’s SharePoint Blog
• Display Dynamic Messages in the Status Bar – Tobias Zimmerman
• Dealing with Background Images in Your Dialog Boxes – Heather Waterman
• How to print SharePoint 2010 page without header and navigation – sharepoint-sandbox.com

Upgrading 2007 Designs

• Upgrading an Existing Master Page to the SharePoint Foundation Master Page – MSDN

Ribbon

• Some SharePoint 2010 User Interface Terms – Mindsharp
• SharePoint 2010 developer – UI Enhancements – Ted Pattison
• Architecture of the Ribbon – msdn
• Customizing the Ribbon – msdn
• Customizing the Ribbon – Part I– Chris O’Brien Sharepointnutsandbolts.com
• Ribbon Customization Part 2 – Adding Items – Chris O’Brien Sharepointnutsandbolts.com
• Ribbon customizations – dropdown controls, Client Object Model and JavaScript Page Components – Chris O’Brien Sharepointnutsandbolts.com
• Enabling a Button on the Ribbon Based on Selection – Blogs.MSDN.com/SharePoint
• Creating a Custom Ribbon Tab in SharePoint 2010 – SharePoint Solutions.com
• Un-Floating the Ribbon in custom Master Pages – Randy Drisgill
• Ribbon Customization: Changing Placement, Look and Behavior – Tom Wilson, Styled Point
• Customizing Ribbon Positioning – Jonathan Kern UX Developer, SharePoint Foundation
• Code to Hide the Ribbon and Site Actions Menu for Anonymous Users – Elumination (Useful for public facing SharePoint sites)

Mobile

• Blog Post: SharePoint 2010 Mobile Devices and Scrolling
• Branding Tip #6 – Mobile Views

Templates, Tools & Themes

• 2 Free templates for SharePoint 2010 Publishing sites
• SharePoint 2010 Themes

SharePoint Design Galleries

• Top SharePoint
• WSS Demo (Live Pivot View… Very cool)

Videos

• Creating complex SharePoint 2010 ribbon elements – Yaroslav Pentsarskyy
• Getting started with SharePoint 2010 ribbon – Yaroslav Pentsarskyy

Developer Links

• Customize the ribbon programmatically from web parts and field controls – (sharepointnutsandbolts .com – Chris Obrien)
• Creating a SharePoint 2010 Ribbon extension – part 1 (Wictor Wilen)
• Creating a SharePoint 2010 Ribbon extension – part 2 (Wictor Wilen)

Web Analytics in SharePoint 2010

As part of SharePoint Server 2010, there is a new set of features to help you collect, report, and analyze the usage and effectiveness of your SharePoint 2010 deployment – whether it’s used as an internal or external web portal, a collaboration tool or a document and records management repository.  These features are part of the Web Analytics capabilities of SharePoint 2010.

This blog post will give you more insight into the enhanced Web Analytics features that built into SharePoint 2010. This post will provide an overview of the new Web Analytics features.

Overview

 

Web Analytics Reports

In SharePoint 2010, they improved the set of Web Analytics reports that are available out-of-the-box, which will provide insights into the behavior of users of your SharePoint sites.  There are three categories of reports that you will find:

• Traffic reports: These reports provide metrics such as:
1. How much traffic your site gets (Number of Page Views);
2. Who visits your sites (Top Visitors);
3. How visitors arrive at your site (Top Referrers);
4. Daily Unique Visitors, Top Destinations, Top Browsers, etc;
• Search reports: These reports give you insight into what users are searching for, for example:
1. How many times users searched (Number of Queries);
2. What were the most used search terms (Top Queries);
3. What queries have high failure rates (Failed Queries);
4. Best Bet Usage, Search keywords, etc;
• Inventory reports: These reports display key metrics regarding the inventory of your sites:
1. What is the total disk drive space user (Storage Usage);
2. How many sites exist (Number of Sites);
3. Top Site Product Versions, Top Site Languages, etc;

These reports aggregated at the following levels:

1. Per web application in the farm
2. Per site collection
3. Per site
4. Per search service application

Out-of-the-box, these reports are visible to Administrators at each level.  For example, site-level reports are available to Site Administrators of those sites.  We have also added a new permission level, “View Web Analytics Data,” that will allow users to access these reports without having to give them Administrator privileges.

You can access Web Analytics reports by going to Site Actions -> Site Settings.  Under the Site Actions heading you will see two links, Site Web Analytics Reports and Site Collection Web Analytics Reports. 

When you click on either link, you are taken to an overview page that shows you key metrics for your site.  You can then drill down to other reports by clicking on them on the left navigation. You can also change the date range for the reports by clicking on the Analyze tab on the Ribbon.

Custom Web Analytics Reports

The out-of-the-box reports are useful to get a general understanding of what is happening on your sites.  However, they made it easy for you to get a deeper level of analysis, or to simply create your own reports.  To get started, click on the Customize Report button under the Analyze tab in the Ribbon.  Clicking this button will export the data contained in this report to Excel.  Excel is a power analytics tools and makes it easy for non technical users to add your own charts, set specific filters, and combine data from multiple reports.  In addition, the data within Excel is refreshable, which means that, once you customize the report, it will always be up-to-date with the latest data.  To get more details on the great new features in Excel 2010 for building charts, reports and pivot tables, take a look at the Excel Team blog.

Web Analytics Workflows

Web Analytics Workflows is a powerful new feature set that enables you to get reports sent out either on a schedule or when specific conditions are met.  For example, you can set them up to receive an email every time the total number of pages views drop by 80% week over week.

To setup a Web Analytics Workflow, go to the Web Analytics report that you are interested in and click on Schedule Alerts or Reports on the Analyze tab in the Ribbon.

Clicking this button will guide you through a series of steps to create your Workflow.

Best Bets Suggestions

Best Bets allow Search Administrators to determine what the most relevant search result is for a given keyword. Up until now, Search Administrators had to look at different reports and data to determine which best bets needed to be added. That process is no longer necessary as SharePoint 2010 periodically sends out suggestions for new Best Bets using all the search metrics it has collected. Now, Search Administrators can simply look through each of the Best Bet suggestions and easily accept or reject them.

To access the Best Bet Suggestions, go to Site Actions, click on Site Collection Web Analytics Reports, and the click on Best Bets Suggestions on the left navigation.

Web Analytics Web Part

They created a new web part, the Web Analytics Web Part, targeted at Site Managers. This new Web Part is an end-user facing Web Part that can be easily inserted into any page on your site.  It can be configured to display the ‘most viewed content’ or the ‘most frequent search queries’ in the site. The data in the Web Part is continuously refreshed as new content or new search queries become more popular.

To use this Web Part, go into the Edit mode of one of your Site Pages and click on any place you can add a Web Part.  Then, from the Insert tab on the Ribbon, click on Web Part.  Finally, click on the Content Rollup category and select the Web Analytics Web Part.

After you have inserted the Web Analytics Web Part, you can then configure it to display the data you are interested in.

Conclusion

Using the new Web Analytics features in SharePoint 2010, you will be able to get a deeper understanding of what users are doing, what they want from your site and how you can tailor the SharePoint experience to bets meet their needs.   Keep an eye out for future posts where we will delve deeper into each of the features mentioned above.