Fixing Index Fragmentation in SQL Server 2005 and SQL Server 2008

SolutionFragmented indexes can be de-fragmented in two ways depending on the level of fragmentation and size of the tables in terms of rows.

• Reorganize Index - The reorganize operation doesn't take a lot of system resources and can be done while users are accessing the table that the index exists on, that's the reason it is an "online" operation. The reorganize process reorganizes the leaf nodes of the index physically to match it with the logical order, this physical order matching the logical order improves the performance of index scans. The indexes can be reorganized using ALTER INDEX...REORGANIZE command (more details on this command and its usage are given below). This command is the same as DBCC INDEXDEFRAG of previous versions (though it still available for backward compatibility), but there is one difference, it has some additional features like large objects compaction (LOB_COMPACTION) and it is an online operation. Also, reorganize does NOT update statistics.

Script : Index Reorganize

ALTER INDEX { index_name | ALL } ON

• Rebuild Index - Rebuilding an index means dropping an existing index and creating a new one altogether. Any fragmentation that was in the older index is gone with the drop and in the new index the logical order matches the physical order. Because the existing index is removed and created afresh, users would not be able to access the table and performance may degrade while the index is being built. (If the index you are rebuilding is a cluster index, then your tables will be unavailable to the user during the rebuild process though if it is non-clustered index only a shared lock will be placed on the table so users will be able to access the table, but would not be allowed to do modifications on it). In other words, users will not be able to lock the table during the time the index rebuild is occurring. This could be a major performance bottleneck when rebuilding indexes. Though beginning with SQL Server 2005 you can rebuild your indexes in an online fashion, so that other users are able to access the table and index while the rebuild is being performed. Since you can access the indexes during the rebuild operation, you are not limited to only rebuilding indexes during off-peak hours, though there is some trade-off needed as discussed in the section below.

You might be wondering how SQL Server makes available the index while it is being rebuilt. Well, it uses Row versioning behind the scenes. When you create an index online, it does not drop the existing index and create a new one, but rather it keeps the original index (source) to be used by the users and creates an index (target) parallel to it. The target index resembles the source index which is now a write-only index. Any modifications are applied to the target as well as on the source index. But during the rebuild only the source index will be available for use by the users. Once the index rebuild operation is complete, the original (source) index will be dropped and the meta-data will be updated to make the newly created (target) available for read and write operations. Of course when you plan to rebuild your indexes online, you need to consider its trade-off of having almost double the storage size for indexes to maintain two indexes (source and target) simultaneously (also additional temporary disk might be required for sorting operations).
Indexes can be rebuilt using these two techniques:

• CREATE INDEX Command - One way is to simply drop the index using a DROP INDEX statement followed by a CREATE INDEX statement. Though you can combine these two commands by using the DROP_EXISTING clause of CREATE INDEX command as given below in the script table. You can use the DROP_EXISTING clause to rebuild the index, add or drop columns, modify options, modify column sort order, or change the partition scheme or filegroup. DROP_EXISTING enhances performance when you re-create a clustered index, with either the same or different set of keys, on a table that also has non-clustered indexes. DROP_EXISTING replaces the execution of a DROP INDEX statement on the old clustered index followed by the execution of a CREATE INDEX statement for the new clustered index. The benefit it gives, the non-clustered indexes are rebuilt only once, and even then only if the index definition has changed. With this command you can rebuild the index online.

Script : Index Rebuild using CREATE INDEX Command

ALTER INDEX { index_name | ALL } CREATE [ UNIQUE ] [ CLUSTERED | NONCLUSTERED ] INDEX index_name ON

• Using ALTER INDEX command - This statement replaces the DBCC DBREINDEX statement. The ALTER INDEX statement allows for the rebuilding of the clustered and non-clustered indexes on the table. The drawback with this statement is that you cannot change the index definition the way you can do with the CREATE INDEX command. Though with this command you can rebuild the index online. ALTER INDEX cannot be used to repartition an index or move it to a different filegroup. This statement cannot be used to modify the index definition at all, such as adding or deleting columns or changing the column order. Use CREATE INDEX with the DROP_EXISTING clause to perform these operations as stated above.

Script : Index Rebuild using ALTER INDEX Command

ALTER INDEX { index_name | ALL } ON

Notes

• Rebuilding the Clustered Index (CI) does not rebuild the Non-Clustered Indexes (NCI) unless the keyword ALL is specified, you will have do it separately.
• If you rebuild your indexes online, you must ensure that you have enough available disk space to hold the index that is being created along with the pre-existing index (source and target indexes simultaneously). After the rebuild operation, the old index will be dropped though. Also, rebuilding indexes online takes a significant amount more time and resources than just rebuilding the index. This is usually a considerable tradeoff since the table will remain available during the rebuild operation.
• Also the underlying table cannot be altered, truncated, or dropped while an online index operation is in process.
• For partitioned indexes built on a partition scheme, you can use either of these methods (Reorganize and Rebuild) on a complete index or on a single partition of an index.
• In general, fragmentation on small indexes is often not controllable, in other words rebuilding or reorganizing small indexes often does not reduce fragmentation. That is because the pages of small indexes are stored on mixed extents. Mixed extents are shared by up to eight different objects, so the fragmentation in a small index might not be reduced after reorganizing or rebuilding the index.
• Index rebuilding can be either online or offline.
• Index reorganizing is always executed online.
• The create index operation can be minimally logged if the database recovery model is set to either bulk-logged or simple.
• An index cannot be reorganized or rebuilt if the filegroup in which it resides is offline or set to read-only.

---

Example Script to Rebuild / Reorganize Fragmented Indexes

Let's see an example, the script provided below reorganizes and rebuilds the indexes if the fragmentation level is higher than the given threshold. You can define the threshold for reorganizing as well as for rebuilding and the script will work accordingly. Please note this script may take several minutes to several hours depending on the level of fragmentation and size of the tables in terms of rows. Kindly test it on your dev or test environment before executing on your production environment.

--Description : This script reorganizes and rebuilds the index if the fragmentation level is higher the given threshold
-- You can define the threshold for reorganize as well as for rebuild and script will work accordingly
-- INPUTS : @fillfactor - While rebuilding index what would be FILLFACTOR for new index
-- @FragmentationThresholdForReorganizeTableLowerLimit - Fragmentation Level lower threshold to check for reorganizing the table, if the fragmentation is higher than this level, it will be considered for reorganize
-- @@FragmentationThresholdForRebuildTableLowerLimit - Fragmentation Level lower threshold to check for rebuilding the table, if the fragmentation is higher than this level, it will be considered for rebuild
-- NOTES : PRINT statements are all queued up and don't show up until the entire script is printed. However, there is an alternative to PRINTing messages.
-- You can raise an error that isn't really an error (code of 0) and you'll get the same effect--message will be printed immediately.

DECLARE @cmd NVARCHAR(1000)
DECLARE @Table VARCHAR(255)
DECLARE @SchemaName VARCHAR(255)
DECLARE @IndexName VARCHAR(255)
DECLARE @AvgFragmentationInPercent DECIMAL
DECLARE @fillfactor INT
DECLARE @FragmentationThresholdForReorganizeTableLowerLimit VARCHAR(10)
DECLARE @FragmentationThresholdForRebuildTableLowerLimit VARCHAR(10)
DECLARE @Message VARCHAR(1000)

SET NOCOUNT ON

--You can specify your customized value for reorganize and rebuild indexes, the default values
--of 10 and 30 means index will be reorgnized if the fragmentation level is more than equal to 10
--and less than 30, if the fragmentation level is more than equal to 30 then index will be rebuilt
SET @fillfactor = 90
SET @FragmentationThresholdForReorganizeTableLowerLimit = '10.0' -- Percent
SET @FragmentationThresholdForRebuildTableLowerLimit = '30.0' -- Percent

BEGIN TRY

-- ensure the temporary table does not exist
IF (SELECT OBJECT_ID('tempdb..#FramentedTableList')) IS NOT NULL
DROP TABLE #FramentedTableList;

SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Retrieving indexes with high fragmentation from ' + DB_NAME() + ' database.'
RAISERROR(@Message, 0, 1) WITH NOWAIT

SELECT OBJECT_NAME(IPS.OBJECT_ID) AS [TableName], avg_fragmentation_in_percent, SI.name [IndexName],
schema_name(ST.schema_id) AS [SchemaName], 0 AS IsProcessed INTO #FramentedTableList
FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL , NULL) IPS
JOIN sys.tables ST WITH (nolock) ON IPS.OBJECT_ID = ST.OBJECT_ID
JOIN sys.indexes SI WITH (nolock) ON IPS.OBJECT_ID = SI.OBJECT_ID AND IPS.index_id = SI.index_id
WHERE ST.is_ms_shipped = 0 AND SI.name IS NOT NULL
AND avg_fragmentation_in_percent >= CONVERT(DECIMAL, @FragmentationThresholdForReorganizeTableLowerLimit)
ORDER BY avg_fragmentation_in_percent DESC

SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Retrieved indexes with high fragmentation from ' + DB_NAME() + ' database.'

RAISERROR(@Message, 0, 1) WITH NOWAIT
RAISERROR('', 0, 1) WITH NOWAIT

WHILE EXISTS ( SELECT 1 FROM #FramentedTableList WHERE IsProcessed = 0 )
BEGIN

  SELECT TOP 1 @Table = TableName, @AvgFragmentationInPercent = avg_fragmentation_in_percent,
  @SchemaName = SchemaName, @IndexName = IndexName
  FROM #FramentedTableList
  WHERE IsProcessed = 0

  --Reorganizing the index
  IF((@AvgFragmentationInPercent >= @FragmentationThresholdForReorganizeTableLowerLimit) AND (@AvgFragmentationInPercent < @FragmentationThresholdForRebuildTableLowerLimit))
  BEGIN
    SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Reorganizing Index for [' + @Table + '] which has avg_fragmentation_in_percent = ' + CONVERT(VARCHAR, @AvgFragmentationInPercent) + '.'
    RAISERROR(@Message, 0, 1) WITH NOWAIT
    SET @cmd = 'ALTER INDEX ' + @IndexName + ' ON [' + RTRIM(LTRIM(@SchemaName)) + '].[' + RTRIM(LTRIM(@Table)) + '] REORGANIZE'
    EXEC (@cmd)
    --PRINT @cmd
    SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Reorganize Index completed successfully for [' + @Table + '].'
    RAISERROR(@Message, 0, 1) WITH NOWAIT
    RAISERROR('', 0, 1) WITH NOWAIT
  END
  --Rebuilding the index
  ELSE IF (@AvgFragmentationInPercent >= @FragmentationThresholdForRebuildTableLowerLimit )
  BEGIN
    SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Rebuilding Index for [' + @Table + '] which has avg_fragmentation_in_percent = ' + CONVERT(VARCHAR, @AvgFragmentationInPercent) + '.'
    RAISERROR(@Message, 0, 1) WITH NOWAIT
    SET @cmd = 'ALTER INDEX ' + @IndexName + ' ON [' + RTRIM(LTRIM(@SchemaName)) + '].[' + RTRIM(LTRIM(@Table)) + '] REBUILD WITH (FILLFACTOR = ' + CONVERT(VARCHAR(3),@fillfactor) + ', STATISTICS_NORECOMPUTE = OFF)'
    EXEC (@cmd)
    --PRINT @cmd
    SET @Message = 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' - Rebuild Index completed successfully for [' + @Table + '].'
    RAISERROR(@Message, 0, 1) WITH NOWAIT
    RAISERROR('', 0, 1) WITH NOWAIT
  END

  UPDATE #FramentedTableList
  SET IsProcessed = 1
  WHERE TableName = @Table
  AND IndexName = @IndexName
END

DROP TABLE #FramentedTableList

END TRY

BEGIN CATCH
  PRINT 'DATE : ' + CONVERT(VARCHAR, GETDATE()) + ' There is some run time exception.'
  PRINT 'ERROR CODE : ' + CONVERT(VARCHAR, ERROR_NUMBER())
  PRINT 'ERROR MESSAGE : ' + ERROR_MESSAGE()
END CATCH