Tag Archives: sql 2014

Filtered index gotcha

We added a filtered index to a table today and a SQL Agent job that updates that table started to fail with this error message:

Msg 1934, Sev 16, State 1, Line 290 : UPDATE failed because the following SET options have incorrect settings: ‘QUOTED_IDENTIFIER’. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes and/or query notifications and/or XML data type methods and/or spatial index operations. [SQLSTATE 42000]

It turns out that you must have certain options enabled in your session in order to use or update a filtered index.

All these must be ON:

  • ANSI_NULLS
  • ANSI_PADDING
  • ANSI_WARNINGS
  • ARITHABORT
  • CONCAT_NULL_YIELDS_NULL
  • QUOTED_IDENTIFIER

And this one must be OFF:

  • NUMERIC_ROUNDABORT

See Books Online for details: https://msdn.microsoft.com/en-us/library/ms188783.aspx (it’s about half way down the page in the Remarks section).

So what happened here? Well it looks like SQL Agent does not, by default, set QUOTED_IDENTIFIER on. I can’t find any clear official information on this but that’s what I’m seeing in SQL Profiler. I also couldn’t find any way to override this in SQL Agent’s configuration settings.

Anyway, I added SET QUOTED_IDENTIFIER ON at the start of the job step and it worked fine.

Handling service failures

There are times when a SQL-related service has a problem and just stops. Windows services are like that sometimes. Unless you do some additional configuration, you’re not going to know about it until users start shouting at you. Here’s what you can do to make that less likely.

Open up Services and double click on the SQL Server service. By default the Recovery tab looks like this:

20160304_default_recovery_tab

If this services stops unexpectedly you won’t know. Not until your phone starts ringing off the hook!

Here’s how I suggest you improve things:

20160304_configured_recovery_tab

The first time the service fails the system will wait a minute and try to restart it. A second failure will get the same treatment. If it fails a third time you need to know so the third failure runs a PowerShell script to send an alert email.

This won’t kick in if the service is stopped by hand or by something like a SQL Server service pack. It’s only for when things go wrong.

The full contents of the Command line parameters box goes like this:

-file "\\server_name\share_name\email_service_crash.ps1" -ServerName "server_name" -ServiceName "SQL Server"

Clearly you’ll need to edit the file path, server name and service name to suit your environment.

The powershell script, email_service_crash.ps1, is this:

Param (
    [Parameter(Mandatory=$True)]
    [String]$ServerName,

    [Parameter(Mandatory=$True)]
    [String]$ServiceName
)

$from = "from_address"
$to = "to_address"
$subject = $ServiceName + " service is down on " + $ServerName
$body = "Find out why and get it started again."

$creds = new-object System.Net.NetworkCredential("smtp_user_name", "smtp_user_password")
$smtp = new-object System.Net.Mail.SmtpClient("smtp_server_name")
$smtp.UseDefaultCredentials = "False"
$smtp.Credentials = $creds
$smtp.Send($from, $to, $subject, $body)

Again, you’ll need to edit some of the variables and parameters: $from, $to, and the SMTP server, user and password.

Set the script up and test it in a cmd window like this:

powershell.exe -file "\\server_name\share_name\email_service_crash.ps1" -ServerName "server_name" -ServiceName "SQL Server"

Once you’ve got it working, you can take the string from ‘-file’ onwards and use it in the service Recovery tab.

You might be asking what the checkbox Enable actions for stops with errors means. Good question. The documentation is no help but it turns out there are some situations where this stuff won’t kick in unless that box is ticked. For more details have a look at this question on Server Fault: http://serverfault.com/questions/72318/set-up-recovery-actions-to-take-place-when-a-service-fails

You can re-use this in the Recovery tab for each service you want to cover. Just remember to change the -ServiceName parameter you pass to the PowerShell script.

SQL Server service startup failure

I just had a SQL Server service that didn’t start after a reboot of the machine. Error message: “The service did not start due to a logon failure. [0x8007042d]”. Hmm.

In the Local Security Policy editor the service account was no longer listed under the user rights assignment for Logon as a service. I discovered I had no permission to change that because it’s managed by a domain policy.

A search found this thread which helped me to sort it out. I tried resetting the service account from the SQL Server Configuration Manager and that didn’t fix the Logon as a service rights so I still had the same error. But when I reset the service account from Services in the Control Panel instead that re-applied the Logon as a service rights and the service started.

I still have no idea why that permission dropped off but at least now I know how to get it back again. It’s progress of a sort.

Developer database permissions

Here’s a suggestion on how you can grant developers plenty of permissions on a dev or test database without just adding them to the db_owner role (which is going too far).

I found a post by Brent Ozar talking about podcasts he likes. Thanks for the tips, Brent. I checked them out and I particularly like the SQL Data Partners podcast. Thanks for the excellent listening, Carlos. I’m working through the episodes and episode 8 ‘The Principal of Least Privilege’ with Robert Verrell (aka SQL Cowbell) really got me thinking.

Robert talked about why adding developers to the db_owner role is a bad idea, even on a dev server. Then he proposed adding them to a custom high-powered database role, db_developer, instead. This way they have enough rights to do things they need to but hopefully they can’t destroy the server. He blogged about it here.

I like the idea but we regularly refresh our our test databases by restoring from production backups and sanitising sensitive data. This process would undo the db_developer role so I wrapped it up in a stored procedure to make it easy to re-run on each relevant database at the end of the refresh job:

if exists (	select 	1
		from 	dbo.sysobjects
		where 	id = object_id(N'dbo.dba_AddDBDeveloperRole')
			and objectproperty(id, N'IsProcedure') = 1    )
	drop proc dbo.dba_AddDBDeveloperRole;
go

set ansi_nulls on;
set quoted_identifier on;
go

create proc dbo.dba_AddDBDeveloperRole
(
	@DBName		sysname
	, @Login	sysname
	, @DebugMode	bit	= 0
)
as

/*
---------------------------------------------------------------------------------------------------------------------------------------------------
Version : 1.01
Date    : 22/12/2015

Grant developers lots of permissions without the really dangerous ones like db_owner or sysadmin.
Based on this blog post: http://sqlcowbell.com/wordpress/why-nobody-ever-needs-the-db_owner-role/.

Original script from that post:

CREATE ROLE [db_developer] AUTHORIZATION [dbo]
 GRANT ALTER ANY APPLICATION ROLE TO [db_developer]
 GRANT ALTER ANY ASSEMBLY TO [db_developer]
 GRANT ALTER ANY DATABASE DDL TRIGGER TO [db_developer]
 GRANT ALTER ANY DATASPACE TO [db_developer]
 GRANT ALTER ANY FULLTEXT CATALOG TO [db_developer]
 GRANT ALTER ANY MESSAGE TYPE TO [db_developer]
 GRANT ALTER ANY SCHEMA TO [db_developer]
 GRANT CREATE AGGREGATE TO [db_developer]
 GRANT CREATE ASSEMBLY TO [db_developer]
 GRANT CREATE DATABASE DDL EVENT NOTIFICATION TO [db_developer]
 GRANT CREATE DEFAULT TO [db_developer]
 GRANT CREATE FULLTEXT CATALOG TO [db_developer]
 GRANT CREATE FUNCTION TO [db_developer]
 GRANT CREATE PROCEDURE TO [db_developer]
 GRANT CREATE ROLE TO [db_developer]
 GRANT CREATE RULE TO [db_developer]
 GRANT CREATE SCHEMA TO [db_developer]
 GRANT CREATE SERVICE TO [db_developer]
 GRANT CREATE SYNONYM TO [db_developer]
 GRANT CREATE TABLE TO [db_developer]
 GRANT CREATE TYPE TO [db_developer]
 GRANT CREATE VIEW TO [db_developer]
 GRANT CREATE XML SCHEMA COLLECTION TO [db_developer]
 GRANT DELETE TO [db_developer]
 GRANT EXECUTE TO [db_developer]
 GRANT INSERT TO [db_developer]
 GRANT REFERENCES TO [db_developer]
 GRANT SELECT TO [db_developer]
 GRANT SHOWPLAN TO [db_developer]
 GRANT UPDATE TO [db_developer]
 GRANT VIEW DATABASE STATE TO [db_developer]
 GRANT VIEW DEFINITION TO [db_developer]
---------------------------------------------------------------------------------------------------------------------------------------------------
*/

set nocount on;

declare @SQL nvarchar(4000);

print 'Database: ' + @DBName;

print 'Create role';
set @SQL =	'use [' + @DBName + ']'
		+ ' if not exists (select 1 from sys.database_principals where type_desc = ''DATABASE_ROLE'' and name = ''db_developer'')'
		+ ' create role [db_developer] authorization [dbo]';
if @DebugMode = 1
	print @SQL;
else
	exec sp_executesql @SQL;

print 'Grant permissions to the role';
set @SQL =	'use [' + @DBName + ']'
		+ ' grant alter any application role to [db_developer];'
		+ ' grant alter any assembly to [db_developer];'
		+ ' grant alter any database ddl trigger to [db_developer];'
		+ ' grant alter any dataspace to [db_developer];'
		+ ' grant alter any fulltext catalog to [db_developer];'
		+ ' grant alter any message type to [db_developer];'
		+ ' grant alter any schema to [db_developer];'
		+ ' grant create aggregate to [db_developer];'
		+ ' grant create assembly to [db_developer];'
		+ ' grant create database ddl event notification to [db_developer];'
		+ ' grant create default to [db_developer];'
		+ ' grant create fulltext catalog to [db_developer];'
		+ ' grant create function to [db_developer];'
		+ ' grant create procedure to [db_developer];'
		+ ' grant create role to [db_developer];'
		+ ' grant create rule to [db_developer];'
		+ ' grant create schema to [db_developer];'
		+ ' grant create service to [db_developer];'
		+ ' grant create synonym to [db_developer];'
		+ ' grant create table to [db_developer];'
		+ ' grant create type to [db_developer];'
		+ ' grant create view to [db_developer];'
		+ ' grant create xml schema collection to [db_developer];'
		+ ' grant delete to [db_developer];'
		+ ' grant execute to [db_developer];'
		+ ' grant insert to [db_developer];'
		+ ' grant references to [db_developer];'
		+ ' grant select to [db_developer];'
		+ ' grant showplan to [db_developer];'
		+ ' grant update to [db_developer];'
		+ ' grant view database state to [db_developer];'
		+ ' grant view definition to [db_developer];';
if @DebugMode = 1
	print @SQL;
else
	exec sp_executesql @SQL;

print 'Create a user for the login';
set @SQL =	'use [' + @DBName + ']'
		+ ' if not exists (select 1 from sys.database_principals where type_desc = ''SQL_USER'' and name = ''' + @Login + ''')'
		+ ' create user [' + @Login + '] for login [' + @Login + '] with default_schema = [dbo];';
if @DebugMode = 1
	print @SQL;
else
	exec sp_executesql @SQL;

print 'Add user to role';
set @SQL =	'use [' + @DBName + ']'
		+ ' if not exists ('
		+ 'select 1'
		+ ' from sys.database_role_members rm'
		+ ' inner join sys.database_principals r on r.principal_id = rm.role_principal_id'
		+ ' inner join sys.database_principals m on m.principal_id = rm.member_principal_id'
		+ ' where r.type_desc = ''DATABASE_ROLE'''
		+ ' and r.name = ''db_developer'''
		+ ' and m.name = ''' + @Login + ''''
		+ ')'
		+ ' alter role [db_developer] add member [ITDev];';
if @DebugMode = 1
	print @SQL;
else
	exec sp_executesql @SQL;

go

The proc is re-runnable and only does the bits that need to be done. It doesn’t check each permission because you don’t get an error if it’s already granted.

Note that the actual permissions bit is just a list so you could easily comment out any you don’t want to grant. Indeed you really should look at all these permissions and see what they mean.

If you set @DebugMode = 1 it will just print out the SQL instead of executing it. This was very handy during development and you could use it if you change anything or if you want to run the changes past somebody for compliance checking.

You can call it for a specific database like this:

exec master.dbo.dba_AddDBDeveloperRole
	@DBName = 'DatabaseX'
	, @Login = 'LoginY';

Or you could apply it to all user databases like this:

print 'Grant rights to DevTeam'

declare @DBName sysname;

declare db_cursor cursor
local forward_only
for
select	name
from	sys.databases
where	name not in ('master', 'model', 'msdb', 'tempdb')
	and state_desc = 'ONLINE'
	and is_read_only = 0
order by name;

open db_cursor;

	fetch next from db_cursor
	into 	@DBName;

	while @@fetch_status = 0
	begin
		exec master.dbo.dba_AddDBDeveloperRole
			@DBName = @DBName
			, @Login = 'DevTeam';

		fetch next from db_cursor
		into 	@DBName;
	end

close db_cursor;
deallocate db_cursor;

Tweak the WHERE clause to exclude other databases.

You can download the code here.

Reporting and the Edge browser

I recently updated my dev machine to Windows 10. I like it a lot more than Windows 8.1. I was keen to see whether Reporting works in the new Edge browser and it turns out it works just fine. Apart from a strange inability to edit a data source in Report Manager.

When you update anything on a data source you have to re-enter the login’s password but in Edge the page keeps insisting the password is wrong and refuses to save the change.

So I recommend you edit data sources in Internet Explorer for now.

PowerShell script path limitations

Having discovered that SQL Agent PowerShell steps use PowerShell 2.0, I found you can use the latest version of PowerShell on your server by invoking PowerShell and passing it the path to a script file:

powershell.exe -file “\\servername\path\filename.ps1”

I thought it would make the job easier to port to a new server one day if, instead of \\servername, I used \\127.0.0.1 but PowerShell said that the current execution policy prevented running remote unsigned scripts. (Our execution policy was RemoteSigned.)

I don’t understand why it treats the names this way. According to my reading, you can spoof \\machinename simply by adding it to the hosts file and redirecting it elsewhere. But, although you can do that with 127.0.0.1, it doesn’t work so I would have though PowerShell would treat such a script as local.

I’m not sure what’s going on here but it’s one to remember.

 

SQL 2014 upgrade experience

I upgraded our servers to SQL 2014 and found it a pretty smooth experience. Thomas LaRock has a nice set of advice here – thanks Tom – with explanations of why and links to further reading.

We started from a mix of mostly SQL 2012 but a few stragglers on SQL 2008R2 that, for various reasons, I hadn’t got round to. I kept the compatibility levels as they were for a while to bed things in. There were no problems at all.

So then I changed the compatibility levels to 120 (SQL 2014) which invokes the new cardinality estimator. As far as I can see, performance is at least as good as before but two query plans did go bad. One needed a little refactoring but the other was very stubborn. After running out of other ideas I used the query hint OPTION (QUERYTRACEON 9481) to force the old cardinality model and that fixed it. I don’t like query hints but in this case I’ll make an exception.

I also notice a distinct improvement in the missing index recommendations in the Performance Dashboard report. For several years it’s been recommending two or three indexes that we already have. Those are gone now and that’s a good sign in my book.