Showing posts with label Development. Show all posts
Showing posts with label Development. Show all posts

Thursday, September 30, 2010

Be Thorough When Making Schema Changes

This morning we made some changes to the database.  We realized that as part of a re-factor we needed to move 2 columns from table A to table B.  We made the changes and immediately a lot of unit tests began failing.  As we worked on it we found out we forgot a couple of things:

  1. The logging trigger on the table needed to be changed.  I take responsibility for this.  Another developer made the change, talked to me, and I didn’t think it through.  This is one reason I use triggers minimally, and why some people won’t use them at all.
  2. We are using Linq to SQL for data access on this project.  I haven’t been shy about stating my dislike for Linq to SQL, but allowed it on this project partly so I could understand it well enough to complain about it intelligently.  Well, when you make changes to a the database schema, those changes need to be made to the DBML as well because Linq to SQL is generating SQL statements from the DBML which means the SQL being generated will be wrong if you don’t update the DBML.

Moral of the story

Make sure you consider everything and take your time when making schema changes.

Monday, February 22, 2010

What’s up with the reads?

So the other day I read Andrew Kelly’s article, Do You Have Hidden Cache Problems, in the latest electronic edition of SQL Server Magazine, and decided to run the query he listed in the article to see if my server was suffering from this problem:

SELECT
SUM(single_pages_kb + multi_pages_kb) AS "CurrentSizeOfTokenCache(kb)"
FROM
sys.dm_os_memory_clerks
WHERE
name = 'TokenAndPermUserStore'


My result was 189344 kb.  In the article Andrew states that:




…if its under 10MB, you should be fine.  I’d start getting concerned between 10MB and 50MB.  If the amount of the consumed memory is over 50MB, you’re probably affected by this problem.




Well, my value converts to nearly 185MB!!!  I guess I must have this problem.  The article says this about the cause:




On a system with reuses query plans effectively, this cache would be as large as only a few megabytes,… But on a 64-bit system with lots of ad hoc or dynamic SQL queries, this cache can grow to hundreds of megabytes in size.




So apparently my lightly loaded server (1 main application on it right now) and a max of about 50 user connections has a lot of ad hoc or dynamic SQL hitting it.  Well, since the application is a third-party application I decided to run this query to see what I had in the procedure cache for ad hoc queries:



SELECT
DEQS.execution_count,
DEQS.total_worker_time,
DEQS.total_physical_reads,
DEQS.total_logical_reads,
DEQS.total_elapsed_time,
DECP.refcounts,
DECP.usecounts,
DECP.cacheobjtype,
DECP.objtype,
DEST.text
FROM
sys.dm_exec_query_stats AS DEQS JOIN
sys.dm_exec_cached_plans AS DECP
ON DEQS.plan_handle = DECP.plan_handle CROSS APPLY
sys.dm_exec_sql_text(DEQS.sql_handle) AS DEST
WHERE
DECP.objtype = 'ADHOC'
ORDER BY
DEQS.execution_count DESC


Hmmm, 4800+ rows returned.  As looked a little deeper I noticed that rows 10-12 based on execution count were the same query with different WHERE clauses.  The queries looked like this:



SELECT TOP 1
Column1
FROM
Table WITH
(NOLOCK)
WHERE
(Column2 = 'Column Default Value' Or
Column2 = 'Another Column Value') AND
(
Column3 = 'Column Default Value') AND
Column4 = 'Value' AND
Column5 = 'Value'
ORDER BY
Column2,
Column3


Interesting.  Then I looked a little further down the list and found some more of the same query.  So I reran my query adding WHERE text Like '’%KeyWord1%KeyWord2%KeyWord3%’ and found over 600 rows for this “single” query in the cache.  Now, those of you who read my blog know that I am a believer in stored procedures for data access and if you aren’t using stored procedures then use parameterized SQL.  This is a great example of a query that needs to be encapsulated into stored procedure or parameterized.



It gets even better.  I took one of the queries and ran it to see what it returned.  It returned nothing! So I ran SELECT * FROM Table to see what was in the table, nothing at least at that time!  The fun part is that running the application query does a Clustered Index Seek, but takes 2 scans to do it according to STATISTICS IO and the SELECT * takes does a Clustered Index Scan but only scans the index once according to STATISTICS IO.  So I looked at the clustered index on the table and it is (based on my query):



    Column2, Column3, Column4, Column5, Column6



So the only column in the table not included in the clustered index is Column1 (the value being returned).  As I looked at the query, it hit me.  There is an OR in the where clause, so in the Query Plan you have 2 SEEK predicates thus 2 scans of the index.  If you remove the OR you get one SEEK predicate and 1 scan.



So what does all this mean?  Well, for me it solidifies 2 things:




  1. Use stored procedures or at the least parameterized SQL.


  2. OR’s are bad and should be avoided as much as possible



So what am I going to do?  I honestly don’t know for sure.  Because there are no complaints about performance I don’t feel the need to address the TokenAndPermUserStore issue and I’m not sure what I’d do anyway besides possibly using Forced Parameterization.  I think I will contact the vendor (they’ve been good to work with so far) to find out what the query really does and to find out why it isn’t parameterized or in a stored procedure since most of their other stuff is.  I’m open to suggestions.

Tuesday, February 16, 2010

SQL Developers Please Use the Table Name/Alias to Prefix Columns

Note:  After I completed this post Aaron Bertrand added this subject to his “Bad Habits to Kick” series for myself and Jay as we suggested on Twitter.  I wrote this post because I knew Aaron was headed to the Winter Olympics so I didn’t think he’d get to it for few weeks and I needed an idea.  Thanks for getting to it, Aaron.
This post is almost an extension of Aaron Bertrand’s (@AaronBertrand) excellent “Bad Habits to Kick” series (if you aren’t reading his blog you should) and was inspired by this tweet:
image
Basically Jay was looking at a query like this (hard for me to write in SSMS because of SQLPrompt):
SELECT
Sales.SalesOrderHeader.SalesOrderID,
OrderDate,
ShipDate,
Status,
Sales.SalesOrderHeader.TerritoryID,
SubTotal,
TaxAmt,
Freight,
TotalDue,
Comment,
OrderQty,
UnitPriceDiscount,
LineTotal,
SP.SalesPersonID,
SP.TerritoryID,
SalesQuota
FROM
Sales.SalesOrderHeader JOIN
Sales.SalesOrderDetail AS SOD
ON Sales.SalesOrderHeader.SalesOrderID = SOD.SalesOrderID JOIN
Sales.SalesPerson AS SP
ON Sales.SalesOrderHeader.SalesPersonID = SP.SalesPersonID

Now, for the original person writing this code it makes perfect sense and they know which column belongs in which table, but, if there are performance issues and you call someone in to help, they won’t know what tables the columns belong to without looking at the schema.  As a matter of fact, the original author likely won’t remember the tables the columns belong to 6 months later either.  In my opinion the above query should look like this:




SELECT
SOH.SalesOrderID,
SOH.OrderDate,
SOH.ShipDate,
SOH.Status,
SOH.TerritoryID,
SOH.SubTotal,
SOH.TaxAmt,
SOH.Freight,
SOH.TotalDue,
SOH.Comment,
SOD.OrderQty,
SOD.UnitPriceDiscount,
SOD.LineTotal,
SP.SalesPersonID,
SP.TerritoryID,
SP.SalesQuota
FROM
Sales.SalesOrderHeader AS SOH JOIN
Sales.SalesOrderDetail AS SOD
ON SOH.SalesOrderID = SOD.SalesOrderID JOIN
Sales.SalesPerson AS SP
ON SOH.SalesPersonID = SP.SalesPersonID

I know without going anywhere else what table each column belongs to and it is consistent.  I even think the Aliases meet with Aaron’s standard.


I know I can be a bit OCD with formatting T-SQL, but I like to be able to read the code quickly and at a glance.  See this post for my standards (I’ve actually slightly changed because SQLPrompt doesn’t, at least not that I’ve found, do EXACTLY what I like).


What do you think?

Thursday, December 10, 2009

Do You Use Projects/Solutions in SSMS?

When SQL Server 2005 was released Microsoft made a major change to the client tools provided with SQL Server, moving from Enterprise Manager for management tasks and Query Analyzer for scripting to the Visual Studio-based SQL Server Management Studio.  I have to admit that when SSMS was released I didn’t like it and continued using Query Analyzer for a while.  I have now adjusted to SSMS and, while I still wish I had a simple query tool, have grown to appreciate what it does.  My two favorite features are the Standard Reports and the ability to add Projects and Solutions like in Visual Studio.
I have to admit that I don’t use the Project and Solutions feature enough in my day to day work, but I do use it when I put together a blog post or presentation for a SQLSaturday or User Group.  By putting all the code for the demos in a Project I don’t have to go searching the file system to find a demo, I don’t have to have the code pre-loaded into SSMS, and I can stay in SSMS throughout the demos.  Here’s my project for my Default Trace presentation:
DefaultTraceProjectAs you can see it provides a single place for all your scripts.  You can also add the Project/Solution to your source control product of choice (I don’t have one installed on my personal laptop at this time). 
I will be working with this feature more to have projects for my database scripts, maintenance scripts, and my favorite queries in addition to using it for presentations and blogs.
I’d love to hear how you are using projects and solutions in SSMS.

Tuesday, December 8, 2009

T-SQL Tuesday #001 – Dates and Times

T-SQL Tuesday was started by Adam Machanic (@AdamMachanic)  on his blog to encourage SQL Bloggers to share their tips and tricks about a specific topic once a month.  A great idea and a great way to get a topic to blog about!
I’ve been working with SQL Server since 1999 and it wasn’t until the last couple of years that I finally learned a better way to retrieve a date range.  I used to do:
SELECT
columns
FROM
TABLE
WHERE
date_column BETWEEN start_date AND end_date;

I’m guessing that there are people reading this that are saying, “I do that all the time and it works fine for me, so what’s the problem?”.  I understand where they are coming from, but I finally really understood that, until the specific DATE data type (date only) in SQL Server 2008, datetime and smalldatetime columns ALWAYS have a time part.  This can and does affect date range queries.  Here’s an example, albeit slightly contrived, but I’ve seen it happen in the real word:

DECLARE @sales TABLE (sale_id INT IDENTITY(1,1) PRIMARY KEY, sale_date DATETIME, sale_amt FLOAT);

WITH cteNums AS 
(
SELECT TOP 50
ROW_NUMBER() OVER (ORDER BY AC.NAME) AS N
FROM
sys.all_columns AS AC 
)
INSERT INTO @sales (
sale_date,
sale_amt
)
SELECT
DATEADD(DAY, -N, GETDATE()) AS sales_date,
N * ABS(CHECKSUM(NEWID()))/10000.00 AS sale_amt 
FROM
cteNums 

/* 
Setup for contrived example.
Make sure the first sale of the month is at midnight 
*/     
UPDATE @sales
SET sale_date = DATEADD(DAY, DATEDIFF(DAY, 0, sale_date), 0)
WHERE
DATEPART(DAY, sale_date) = 1

/*
Task is to get sales for the previous full month
*/
/*
Set up variables for the first and last day of th month
*/
DECLARE @start_date DATETIME,
@end_date DATETIME

/*
Set the variables to the first of last month and the last day of 
last month at the time of writing '2009-11-01' and '2009-11-30' See this blog post by Lynn Pettis for why I am using DATEADD and DATEPART with 0
*/        
SELECT
@start_date = DATEADD(MONTH, DATEDIFF(MONTH, 0, GETDATE())-1, 0),
@end_date = DATEADD(DAY, -1, DATEADD(MONTH, 1, @start_date))

SELECT @start_date, @end_date, DATEADD(DAY, 1, @end_date)

/*
My old method
*/
SELECT
COUNT(*) AS sales,
MIN(sale_date) AS first_sale,
MAX(sale_date) AS last_sale,
SUM(sale_amt) AS total_sales 
FROM
@sales
WHERE
sale_date BETWEEN @start_date AND @end_date 

/*
My new method - accurate
*/    
SELECT
COUNT(*) AS sales,
MIN(sale_date) AS first_sale,
MAX(sale_date) AS last_sale,
SUM(sale_amt) AS total_sales 
FROM
@sales
WHERE
sale_date >= @start_date AND 
/*First of the next month */
sale_date < DATEADD(DAY, 1, @end_date)

Now I'm sure someone out there will say, “Hey in your last example you are using the first day of the next month and you could do that with between.” Well yes I could and here is the query:

SELECT
COUNT(*) AS sales,
MIN(sale_date) AS first_sale,
MAX(sale_date) AS last_sale,
SUM(sale_amt) AS total_sales 
FROM
@sales
WHERE
sale_date BETWEEN @start_date AND DATEADD(DAY, 1, @end_date)

But this does not return the correct results either.  Here are the results for each query:


Count
First Sale
Last Sale
Total
1st Between
29
2009-11-01 00:00:00.000
2009-11-29 11:09:30.107
86362080.49
>= and <
30
2009-11-01 00:00:00.000
2009-11-30 11:09:30.107
86362080.49
2nd Between
31
2009-11-01 00:00:00.000
2009-12-01 00:00:00.000
86362080.49

All that to show that I believe you are better off using >= and < instead of BETWEEN when comparing dates.

Monday, November 30, 2009

Maintaining Security and Performance Using Stored Procedures Part II – Signing

Well, it has been a couple of weeks since my last blog post and over a month since Maintaining Security and Performance Using Stored Procedures Part I – Using EXECUTE AS was posted, although I did spend time working on the originally unplanned follow up to that post when I would have been doing this post.  I know you all have been anxiously awaiting this post.

EXECUTE AS vs. Signing

Like EXECUTE AS, the ability to sign a module (stored procedure, DML trigger, function, or assembly) with a certificate was added in SQL Server 2005.  In addition to being able to allow access to objects within the current database context, signing also allows you to access resources in another database or that require server level permissions.  With EXECUTE AS on functions, stored procedures, and DML triggers you are limited by database context, so if you need access to objects in another database you are out of luck, even if the EXECUTE AS user has proper rights in the database.  You can download code that demonstrates this behavior here.

Demonstrations of Signing

Laurentiu Cristofor has an excellent blog post that demonstrates signing a stored procedure to grant server level permissions here, so I am not going to duplicate his work in this post.  I will demonstrate how to sign a procedure for use within a database and when accessing another database.
Using Signing to enable Dynamic SQL within the database
USE AdventureWorks;
GO

/*
Create a restricted_login
*/
CREATE LOGIN restricted_login WITH Password = '$tr0ngPassword';

GO

/*
Create a Certificate first
*/    
CREATE CERTIFICATE cert_dynamic_sql 
ENCRYPTION BY PASSWORD = 'c3rtificatePa$$w0rd'
WITH subject = 'Dynamic SQL Security'   
GO 

/* 
Create user based on the certificate
*/ 
CREATE USER certificate_user FROM CERTIFICATE cert_dynamic_sql;

/*
Give the certificate_user select on all objects
*/
GRANT SELECT ON SCHEMA::Person TO certificate_user;
GO

/*
Create a restricted rights database user
*/    
CREATE USER restricted_user FROM LOGIN restricted_login;

GO

/*
Create the procedure
*/
CREATE PROCEDURE dbo.FindPhoneByName
(
@LastName nvarchar(50) = null,
@FirstName nvarchar(50) = null
)
AS   
BEGIN
 SET NOCOUNT ON;

Declare @sql_cmd nvarchar(2000),
@select nvarchar(1000),
@where nvarchar(1000),
@parameters nvarchar(1000);

 Set @parameters = N'@FirstName nvarchar(50), @LastName nvarchar(50)';

Set @select = N'Select
Title,
FirstName,
MiddleName,
LastName,
Suffix,
Phone  
From
Person.Contact';

Set @where = N' Where 1=1 '                

If @LastName is not null
Begin
Set @where = @where + N' And LastName Like @LastName + N''%'' ';
End;

If @FirstName is not null     
Begin
Set @where = @where + N' And FirstName Like @FirstName + N''%''';
End;

Set @sql_cmd = @select + @where;

Exec sys.sp_executesql @sql_cmd, @parameters, @LastName = @LastName, @FirstName = @FirstName;

Return;         
END

GO

/*
Give the restricted user exec on the proceduere
*/
GRANT EXEC ON dbo.FindPhoneByName TO restricted_user; 

GO
/*
Change context to the restricted rights user
*/
EXECUTE AS LOGIN = 'restricted_login';

GO

/*
Exec the procedure - will fail on the dynamic portion
*/
EXEC [dbo].FindPhoneByName;

GO

/* 
Return to sysadmin rights
*/
revert;

GO

/*
Sign the procedure
*/
ADD SIGNATURE TO [dbo].[FindPhoneByName]
BY CERTIFICATE cert_dynamic_sql
WITH PASSWORD = 'c3rtificatePa$$w0rd';
GO

/*
Change context to the restricted rights user
*/
EXECUTE AS LOGIN = 'restricted_login';

GO

/*
Exec the procedure - will work now
*/
EXEC [dbo].FindPhoneByName;

GO

/* 
Return to sysadmin rights
*/
revert;

GO
Using Signing to Access an Object in Another Database
In this example I’ll use a signed procedure to access a table in the Northwind database (download here) from the AdventureWorks database (I use a 2005 copy with extra data, the unmodified version is available here).  One thing I found in my testing is that you have to use a private key file in this case.  If anyone knows how to do it without the file please let me know.  Here is the code:

USE MASTER;
GO

/*
Create the restricted_login
*/
CREATE LOGIN restricted_login WITH Password = '$tr0ngPassword';

GO

USE Northwind;

GO

/*
Create the Certificate
*/    
CREATE CERTIFICATE cert_access_other_db 
ENCRYPTION BY PASSWORD = 'c3rtPa$$word'
WITH subject = 'Access Other DB'   
GO 

/* 
Backup the certificate being sure to use a Private Key
*/
BACKUP CERTIFICATE cert_access_other_db TO FILE = 'C:\Certificates\cert_access_other_db.cer'
WITH PRIVATE KEY (FILE = 'C:\Certificates\cert_access_other_db.pvk' ,
ENCRYPTION BY PASSWORD = '3ncRyptKeyPa$$word',
DECRYPTION BY PASSWORD = 'c3rtPa$$word');
GO

/*
Create the certificate user in the Northwind and give needed permissions
*/
CREATE USER certificate_user FROM CERTIFICATE cert_access_other_db;

GO

GRANT SELECT ON dbo.Categories TO certificate_user;

GO 

USE AdventureWorks;

GO

/*
Create a restricted rights database user
*/    
CREATE USER restricted_user FROM LOGIN restricted_login;

GO

/*
Create the procedure
*/
CREATE PROCEDURE [dbo].access_other_db
AS
SET NOCOUNT ON

SELECT 
SYSTEM_USER AS USERName, 
*
FROM
[Northwind].dbo.[Categories] AS C;

RETURN;    

GO

/*
Give the restricted_user execute rights on the sp
*/
GRANT EXEC ON dbo.access_other_db TO restricted_user;

GO

/*
Create the certificate in this database from the file
*/
CREATE CERTIFICATE cert_access_other_db FROM FILE = 'C:\Certificates\cert_access_other_db.cer'
WITH PRIVATE KEY (FILE = 'C:\Certificates\cert_access_other_db.pvk',
DECRYPTION BY PASSWORD = '3ncRyptKeyPa$$word', /*The password used to create the private key */
ENCRYPTION BY PASSWORD = 'D3cryptKeyPa$$word');
GO

/* 
Execute as a sysadmin - works - this is my user
*/
EXEC [dbo].[access_other_db];

GO


/*
Now execute as the restricted user 
*/
EXECUTE AS LOGIN = 'restricted_login';

GO

/*
This will fail.
*/
EXEC [dbo].[access_other_db];

GO

/*
Back to the sysadmin level
*/
Revert;

GO

/*
Sign the procedure 
*/
ADD SIGNATURE TO dbo.access_other_db
BY CERTIFICATE cert_access_other_db WITH Password = 'D3cryptKeyPa$$word'
GO

/*
Now execute as the restricted user 
*/
EXECUTE AS LOGIN = 'restricted_login';

GO

/*
This will now work.
*/
EXEC [dbo].[access_other_db];

GO

/*
Back to the sysadmin level
*/
Revert;

GO

/*
Be sure to delete the certificate and
private key when done.
*/

Summary

As you can see from this post and the previous post (or two), the SQL Server team has given you some good options when it comes to using Stored Procedures for data access and manipulation while still maintaining security.  You can use EXECUTE AS to allow cross-schema or within database access and you can use module signing to allow access to system objects or cross-database queries without specifically granting users access to the objects.

Resources

After having planned and started the post because I had not found anything outside of Books On Line, I found a few resources that covered the material as well, and I used each to help me write this post once I found them.
  • I linked to Laurentiu Cristofor’s post earlier
  • Erland Sommarskog has an excellent write-up on Giving Permissions through Stored Procedures which handles this subject very thoroughly as you would expect form Erland.  Erland’s post helped me get past the fact that I needed to use a private key in order to get the cross database piece working as none of my other resources did this.
  • Jonathan Kehayias also answered this this forum post with an example.
In reality I could have just posted these links as these other folks covered the subject on signing thoroughly, but I decided that since I did the research and wanted to try the code myself that I’d share my experiences as well, crediting these guys for their work and hopefully sending them some traffic, however limited, from this blog.

Finally all the MY code from this post can be downloaded from here.

Monday, October 26, 2009

Book Review: Murach’s JavaScript and DOM Scripting by Ray Harris

In early September I received a complimentary copy of Murach’s JavaScript and DOM Scripting by Ray Harris (Amazon) to review.  I got the book because my friend, Andy Warren, passed my name along to the publisher when they asked him to review the book.  He knew I was attempting to learn JavaScript, so passed along this opportunity to me.

This is the first Murach book I have read and it definitely has an interesting format.  The left page is text and the right page is code, examples, and summary of key points.  It took a couple of chapters to get used to the format, but once I did I found it to be very helpful.  For some of the early chapters I was able to just skim the right page to pick up the concepts as the content was already familiar to me.

The book is broken down into four sections:

  1. Introduction to JavaScript Programming
  2. JavaScript Essentials
  3. Dom Scripting
  4. Other JavaScript Skills

In the first section, Introduction to JavaScript Programming, you get the basics of web development including XHTML, CSS, and beginning JavaScripting.  As an inexperienced web programmer I found this information invaluable.  I’ve always been confused by CSS this book helped me to gain a basic understanding so I can now read, understand, and intelligently edit CSS pages.  An experience web developer will be able to skim/skip much of this section, but as a relative newbie, I ate it all up.

In section two, JavaScript Essentials, you delve deeper into JavaScript functionality.  Getting input and displaying output, working with native objects, control statements, arrays, functions, objects, regex, exception handling, and data validation.  Some of the topics in this section, like control statements, are common to other languages so I was able to skim parts of this section and just use the examples and summary on the right hand page.

In section three, DOM Scripting, you really get into the deeper topics.  This is where you really get to hone your skills and take advantage of the power of the DOM and scripting.  You learn to manipulate the DOM, CSS, and build libraries you can re-use to do this.  You get understandable examples and exercises that lead you through the concepts and help you build working examples of a slide show, manipulating tables, and animations.  From here to the end of the book, I would imagine that even experienced JavaScript developers would learn something.

In section four, Other JavaScript Skills, you learn how to manipulate the browser and leverage existing JavaScript libraries like jQuery to extend your applications.

The book was was easy to read and the examples and exercises were good.  Being a Microsoft developer I was a little disappointed that it the book did not really talk about using JavaScript with Visual Studio, but I guess that is to be expected.  I also didn’t think enough time was spent on showing how to deal with the situation when a user has disabled scripting in their browser.  The last thing was that I found the instructions in some of the exercises were too vague and while I finished them in a manner that worked, I wasn’t sure if I had done it correctly.  The finished scripts were included in the download, but I would have liked to have seen it in the book as an appendix.

So I did have a couple of issues, but overall a really good book.  I think it works for beginners all the way to experienced web developers.  The more you know you can “mine” the parts of the book that address your weak areas.  I’d recommend the book and would buy other Murach books based on my experience with this one.

Thursday, October 8, 2009

Maintaining Security and Performance using Stored Procedures Part I – Using EXECUTE AS

Anyone who knows me, or has worked with me, knows that I am a proponent of using stored procedures for all database access.  I believe that using stored procedures makes your database more secure and makes it simpler to maintain a well performing system.  One area where stored procedures are more difficult to work with than building queries in the GUI or business layer are with dynamic search queries.  Here are some traditional issues with dynamic search in stored procedures:

  1. If you use traditional IF, ELSE statements to build the procedure you get a long and hard read procedure, and you are less likely to get plan re-use.
  2. If you try tricks like WHERE LastName = IsNull(@LastName, LastName) and FirstName = IsNull(@FirstName, FirstName) you can get plan re-use, but the plan used may not be, and many times is not, the best plan to use.
  3. If you use dynamic SQL using the EXEC (@sql) syntax you do not get plan re-use, you open yourself up for SQL Injection, and, prior to SQL Server 2005, you had to grant access to the objects used in the query defeating part of the reason for using stored procedures in the first place.
  4. If you use dynamic SQL using sp_executsql and parameters you are more likely to get plan re-use, you are safe from sql injection, but, pre-2005, you still needed to grant access to the queried objects.
  5. Either dynamic SQL option means creating a large string of SQL and concatenating it, so it can be and, in my opinion is, a pain to read and a pain to make sure you have all your syntax right.

See Erland Sommarskog's excellent articles, The curse and blessings of dynamic SQL and Dynamic Search Conditions, for more details.

Early in my career, when working with SQL Server 7 and 2000 I tended to use option 1, sometimes with temporary tables, then I moved to option 2.  I never used dynamic SQL because I did not want to grant select access to the tables being queried.  I sacrificed performance for security and counted on ownership chaining to handle access to the underlying tables.  With the advent of SQL Server 2005 and the EXECUTE AS I have moved to option 4, dynamic SQL using sp_executesql and parameters as I believe it gives me the best of both worlds.

How’s it work

Essentially you create the stored procedure and add the WITH EXECUTE AS Caller/Owner/Self/’user name’/’login name’ (see the Books on Line entry for EXECUTE AS for more details) and this changes the context in which the code within the procedure is run.  So you can create a user (SelectAll) in the database that has select rights on all the tables and then no matter who calls the stored procedure the procedure will run correctly.  If you choose to use EXECUTE AS OWNER then the procedure executes in the security context of the Owner of the procedure so you can simulate ownership chaining. 

Example

Security

A post like this wouldn’t be complete without at least a simple example.  I will be using the AdventureWorks database (get it at CodePlex, I’m still using the 2005 version) with some added data (I used RedGate SQLDataGenerator).  All the code to run the examples is available for download here.

First you need to create a user with limited persmissions:

Use AdventureWorks;

Go

Create
User DynamicSQLTest without login;


Notice that I used the Without Login syntax so I did not need to create a login as well.  This is because I will also be using EXECUTE AS before running the stored procedure to change my execution context to this limited rights user instead of creating a new connection with a limited rights login.  Next you need to create the stored procedure.  I’m going to start with a “normal” stored procedure using Option 2 from above, because I also want to demonstrate the difference in performance.  Here’s the procedure:



Use AdventureWorks;
GO

IF
OBJECT_ID('dbo.FindPhoneByName', N'P') Is Not Null
Begin
Drop Procedure
dbo.FindPhoneByName;
End;

Go

CREATE PROCEDURE
dbo.FindPhoneByName
(
@LastName nvarchar(50) = null,
@FirstName nvarchar(50) = null
)
AS

BEGIN
SET NOCOUNT ON;

Select
Title,
FirstName,
MiddleName,
LastName,
Suffix,
Phone
From
Person.Contact
Where
LastName Like IsNull(@LastName, LastName) + N'%' And
FirstName Like IsNull(@FirstName, FirstName) + N'%';

Return;
END
GO


This procedure is pretty self explanatory.  Now we need to give the limited rights use, DynamicSQLTest, execute rights on our procedure:



Use AdventureWorks;

Go

Grant Exec on
dbo.FindPhoneByName to DynamicSQLTest;



To test the security and performance of the stored procedure we are going to execute it 3 times with a dbo user and then repeat as the limited rights user, DynamicSQLTest.  Here is what I used:




Exec dbo.FindPhoneByName @FirstName = 'J', @LastName = 'A';

Go

Exec
dbo.FindPhoneByName @FirstName = 'J';

Go

Exec
dbo.FindPhoneByName @LastName = 'A';

Go




Then execute the same 3 calls, but run this first to change the security context:



Execute AS User = 'DynamicSQLTest';

Go



If you are running the code in the same SSMS session be sure to issue the REVERT command to return to your original security context.



The stored procedure calls should run successfully for both users and should produce the same results and performance for both users.  Now we’ll ALTER the procedure to use dynamic SQL:



Alter PROCEDURE dbo.FindPhoneByName
(
@LastName nvarchar(50) = null,
@FirstName nvarchar(50) = null
)
AS

BEGIN
SET NOCOUNT ON
;

Declare @sql_cmd nvarchar(2000),
@select nvarchar(1000),
@where nvarchar(1000),
@parameters nvarchar(1000);

Set @parameters = N'@FirstName nvarchar(50), @LastName nvarchar(50)';

Set @select = N'Select
Title,
FirstName,
MiddleName,
LastName,
Suffix,
Phone
From
Person.Contact'
;

Set @where = N' Where 1=1 '

If @LastName is not null
Begin
Set
@where = @where + N' And LastName Like @LastName + N''%'' ';
End;

If @FirstName is not null
Begin
Set
@where = @where + N' And FirstName Like @FirstName + N''%''';
End;

Set @sql_cmd = @select + @where;

Exec sys.sp_executesql @sql_cmd, @parameters, @LastName = @LastName, @FirstName = @FirstName;

Return;
END



Now when you run our examples, you’ll see that it runs successfully under your original security context, but you receive an error when you run it as the limited rights user:




Msg 229, Level 14, State 5, Line 1




The SELECT permission was denied on the object 'Contact', database 'AdventureWorks', schema 'Person'.




This is because the execution context changed and ownership chaining no longer applies.  To get the Dynamic SQL Stored procedure to work add WITH EXECUTE AS OWNER after the parameter definition like this:



ALTER PROCEDURE dbo.FindPhoneByName
(
@LastName nvarchar(50) = null,
@FirstName nvarchar(50) = null
)
With Execute As owner
AS



Then you can re-run the your stored procedure calls and they should work both for the dbo user and the limited rights user because the EXECUTE AS OWNER has enabled access to the tables.



Perfomance


I ran all of my examples with SET STATISTICS IO ON so I could see the results.  Here are those results (also part of the download):

























































Parameters Non-Dynamic SQL Dynamic SQL
Scans Reads Scans Reads
@FirstName = 'J', @LastName = 'A' 1 593 1 593
@FirstName = 'J' 1 7792 1 1116
@LastName='A' 1 3039 1 1116








Notice the reduced number of reads required by the Dynamic SQL when only 1 parameter is supplied.  This is because it is using a different query plan, while the Non-Dynamic procedure has one query plan which is not optimal when only one parameter is supplied



Conclusions



As you can see some of the limitations of Dynamic SQL have been “cured” by the advent of the EXECUTE AS clause.  This has made it simpler to use Dynamic SQL and get the performance benefits provided by getting a proper execution plan and getting plan re-use.  Again all code is available here.



Next I’ll be discussing using a Certificate to sign a stored procedure.



Part Ib, Part II

Tuesday, October 6, 2009

Index on a Foreign Key?

About two weeks ago, I had a discussion on Twitter and via email with Jeremiah Peschka (@peschkaj) about placing indexes on Foreign Key columns.  I made the comment that I while indexing a foreign key is not required that I thought they made good candidates.  I also shared this blog post by Greg Low with him. 

Jeremiah mentioned that he had never had SQL Server recommend an index on a foreign key which I thought was strange so I decided to run some tests.  I ran the tests against my copy of AdventureWorks to which I have added data.  Here is the query I ran:

SELECT
SOD.SalesOrderDetailID,
SOD.OrderQty,
SOD.ProductID,
SOD.UnitPrice,
SOD.UnitPriceDiscount,
SOD.LineTotal,
SOD.UnitPrice2,
SOH.SalesOrderID,
SOH.RevisionNumber,
SOH.OrderDate,
SOH.DueDate,
SOH.ShipDate,
SOH.Status,
SOH.SalesOrderNumber,
SOH.PurchaseOrderNumber,
SOH.AccountNumber,
SOH.CustomerID,
SOH.SubTotal,
SOH.TaxAmt,
SOH.Freight,
SOH.TotalDue,
SOH.Comment
FROM
Sales.SalesOrderHeader AS SOH JOIN
Sales.SalesOrderDetail AS SOD ON
SOH.SalesOrderID = SOD.SalesOrderID
WHERE
SOH.OrderDate >= '12/1/2007'


In my copy of AdventureWorks I have 131465 rows in SalesOrderHeader and 1121317 rows in SalesOrderDetail.  The Max(SalesOrderHeader.OrderDate) is 12/30/2007 so I just queried for the last month (1095 rows).  In this query SalesOrderDetail.SalesOrderID is a foreign key referencing SalesOrderHeader.SalesOrderID and before modification the Clustered Primary Key on SalesOrderDetail is on SalesOrderID, SalesOrderDetailID.  Because SalesOrderID is the lead column in the clustering key the optimizer uses a clustered index seek to access SalesOrderDetail as shown here:



imageNow, to see if the Optimizer would recommend an index on SalesOrderDetail.SalesOrderID, I altered the clustered primary key on SalesOrderDetail, removing SalesOrderID with the following code:



USE [AdventureWorks]
GO

IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[Sales].[SalesOrderDetail]') AND name = N'PK_SalesOrderDetail_SalesOrderID_SalesOrderDetailID')
ALTER TABLE [Sales].[SalesOrderDetail] DROP CONSTRAINT [PK_SalesOrderDetail_SalesOrderID_SalesOrderDetailID]
GO

USE
[AdventureWorks]
GO

ALTER TABLE [Sales].[SalesOrderDetail] ADD CONSTRAINT [PK_SalesOrderDetail_SalesOrderID_SalesOrderDetailID] PRIMARY KEY CLUSTERED
(
[SalesOrderDetailID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO



I then re-ran my first query and now you can see that I there is a clustered index scan, a hash match join, and, in green at the top, a index recommended on SalesOrderDetail.SalesOrderID:



imageNext I added the suggested index (remember that you should not just add indexes to production without checking other queries as well):



USE [AdventureWorks]
GO
CREATE NONCLUSTERED INDEX
SalesOrderDetails_SalesOrderId
ON [Sales].[SalesOrderDetail] ([SalesOrderID])
INCLUDE ([SalesOrderDetailID],[OrderQty],
[ProductID],[UnitPrice],
[UnitPriceDiscount],[LineTotal],
[UnitPrice2])
GO



Then I re-ran my query, and here’s the execution plan:



image



Here are the Statistics IO results from the 3 queries as well:



Results with SalesOrderID as lead column in Primary Key (Original State)



Table 'SalesOrderDetail'. Scan count 147, logical reads 516, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

Table 'SalesOrderHeader'. Scan count 1, logical reads 461, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.



Results with SalesOrderID removed from Primary Key and not added as an index.



Table 'Worktable'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

Table 'SalesOrderDetail'. Scan count 1, logical reads 12584, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.


Table 'SalesOrderHeader'. Scan count 1, logical reads 461, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.



Results with the suggested index on SalesOrderID.



Table 'SalesOrderDetail'. Scan count 147, logical reads 504, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

Table 'SalesOrderHeader'. Scan count 1, logical reads 461, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.



Conclusion



In this case you can see that adding an index on the foreign key, whether the lead column in the Clustered Primary Key or in a non-clustered index, you get much improved performance particularly regarding reads, and that the non-clustered index actual provides a slightly better access path than being part of the clustered primary key.  Certainly you need to test for your individual situations but, I would say that the majority of the time a foreign key will benefit from an index since it is used as a filter in the JOIN criteria.



AdventureWorks can be downloaded from codeplex.  Execution Plans and T-SQL can be downloaded from here.

Wednesday, September 30, 2009

On vs. Where

Does it matter if you put your criteria in the ON clause or the WHERE clause?  Well, as with most things SQL the answer is, “It depends”.  If you are dealing with INNER JOIN’s then it really doesn’t matter because the query optimizer is smart enough to come up with the same execution plan for both.  For example these 2 queries evaluate to the same execution plan:

SELECT
SOD.SalesOrderDetailID
,
SOH.SalesOrderID
FROM
Sales.SalesOrderHeader AS SOH
JOIN
Sales.SalesOrderDetail AS SOD
ON
SOH.SalesOrderID = SOD.SalesOrderID
WHERE
SOH.OrderDate >= '7/1/2004'
AND
SOH.OrderDate <
'8/1/2004'

SELECT
SOD.SalesOrderDetailID
,
SOH.SalesOrderID
FROM
Sales.SalesOrderHeader AS SOH
,
Sales.SalesOrderDetail AS SOD
WHERE
SOH.SalesOrderID = SOD.SalesOrderID
AND
SOH.OrderDate >= '7/1/2004'
AND
SOH.OrderDate < '8/1/2004'



Execution Plan



The old SQL 6.5 OUTER JOIN syntax (*= and =*) has been discontinued beginning with SQL Server 2005, so you have to do the JOIN for OUTER JOIN’s in the ON as demonstrated in this code:



SELECT

  
SOD.SalesOrderDetailID
,

  
SOH.SalesOrderID


FROM

  
Sales.SalesOrderHeader AS SOH
LEFT JOIN

  
Sales.SalesOrderDetail AS SOD


        ON SOH.SalesOrderID = SOD.SalesOrderID


WHERE

  
SOH.OrderDate >='7/1/2004'
AND

  
SOH.OrderDate <'8/1/2004'





Now let’s create a sandbox to play in.



If OBJECT_ID('sales.orders', 'U') Is Not Null
Begin
Drop Table
sales.orders;
End;

If OBJECT_ID('sales.order_items', 'U') Is Not Null
Begin
Drop Table
sales.order_items;
End;

If SCHEMA_ID('sales') Is Not Null
Begin
Drop Schema
sales;
End;

Go

Create Schema
sales;

Go
/*
Tables to hold sample data
*/
Create Table sales.orders
(
order_id INT IDENTITY(1,1)PRIMARY KEY,
customer_id INT
);

Create Table sales.order_items
(
order_detail_id INT IDENTITY(1, 1)PRIMARY KEY,
order_id INT,
product_id INT,
quantity INT
)

/*
Load Sample data
*/
INSERT INTO sales.orders (customer_id)
SELECT TOP 5
AO.[object_id]
FROM
sys.all_objects AS AO;

INSERT INTO sales.order_items
(
order_id,
product_id,
quantity
)
SELECT
1,
1,
7
Union ALL
Select
2,
1,
4
Union ALL
Select
3,
2,
6
Union ALL
Select
4,
2,
11
Union ALL
Select
5,
3,
1;



Now we want to return all the customers who have placed an order, but we only want to return the items where the quantity is greater than 5.  Here is method 1:


Select
O.customer_id,
OI.order_id,
OI.product_id,
OI.quantity
From
sales.orders AS O LEFT OUTER JOIN
sales.order_items AS OI ON
O.order_id = OI.order_id
Where
OI.quantity > 5;



This returns:



customer_id order_id    product_id  quantity
----------- ----------- ----------- -----------
3 1 1 7
7 3 2 6
8 4 2 11


Hmmm, we know we have orders from five customers, but this only returns the three rows.  Let’s look at the execution plan:



image



What’s that nest loops (inner join) operator?  Well, by putting the criteria for the RIGHT (second) table in the WHERE clause we essentially converted our LEFT OUTER JOIN to an INNER JOIN.  The correct way to get the data we want would be this way:



SELECT

   
O.customer_id,

   
OI.order_id,

   
OI.product_id,

   
OI.quantity

FROM

   
sales.orders AS O LEFT OUTER JOIN

   
sales.order_items AS OI ON

       
O.order_id = OI.order_id AND

       
OI.quantity > 5;



This returns what we would expect to see:





customer_id order_id    product_id  quantity
----------- ----------- ----------- -----------
3 1 1 7
5 NULL NULL NULL
7 3 2 6
8 4 2 11
17 NULL NULL NULL



And here is the execution plan:




image



Where you can see the Nested Loops (left outer join) operator.




So yes, it does matter where you put your criteria when dealing with OUTER JOINS.