CURRENT_TIMESTAMP in SQL Server

CURRENT_TIMESTAMP is a system function to retrieve the current system date and time. CURRENT_TIMESTAMP is an ANSI SQL function whereas GETDATE is the T-SQL version of that same function. That says it is a non-deterministic function. If you are very specific to ANSI comaptibility or thinking of heterogenous system support, I would suggest to use CURRRENT_TIMESTAMP instead of GETDATE(). As an envision, I prefer to use CURRENT_TIMESTAMP over GETDATE() personally wherever possible.

Few of Thoughts:

1. When we create a default constaint with CURRENT_TIMESTAMP, by default, SSMS properties are showing GETDATE().


My thought: This could be because SSMS properties have nothing to do with ANSI compatibility nor hetrogeneous support.

2. Tool tip text of the function adds parenthesis – ()

My thought: I have no clue why Microsoft has made this so.Its more concerned to me the function with “()” gives an error as below.

Advertisements

Always Encrypted – A new column level security feature in SQL Server 2016

Always Encrypted (AE) is a new feature introduced in SQL Server 2016 to secure your data in SQL Server at column level. Perhaps, SQL Server has many options to secure the data, the new feature Always Encrypted stands out from the list with unique characteristics – “Always Encrypted”.

Before we get into details about Always Encrypted, let us quickly look at the security features in the SQL Server in comparison as an overview.

AE-Always Encrypted, DDM – Dynamic Data Masking, TDE – Transparent Data Encryption

Why do we call Always Encrypted?

As the name depicts, Always Encrypted feature in SQL Server always ensures your data encrypted, that means, the data at rest and in motion. The encryption and decryption happens at client application using an Always Encrypted driver. This separates the encryption from SQL Server database engine and enforces more security in a better controlled manner.

How do we implement Always Encrypted?

First and foremost action is to install the right version of SQL Server 2016.If you do not have the right version, you will not find the option “Encrypt Columns” in “Task” of the database options. If you are not using SSMS version 13.0.4001.0 or above, you will not be able to see this option in your SSMS.

You can find and download SP1

There are two ways, we can implement Always Encrypted in SQL Server using Wizard and T-SQL. However, we need to know that for existing table/column data, there is no way to implement the AE using T-SQL in SQL Server 2016(SP1). I mentioned the service pack as a caveat because Microsoft may change this behavior in future, but unlikely as of now. For existing data, we need to make sure the AE needs to be implemented using Wizard. It is quite good to be noted there is an option to generate PowerShell script for the existing data to encrypt that can be run later.

Using Wizard

Using T-SQL


/*1*/ CREATE COLUMN MASTER KEY CMK_Auto2
WITH (  
  KEY_STORE_PROVIDER_NAME = 'MSSQL_CERTIFICATE_STORE',   
  KEY_PATH = 'CurrentUser/my/B27A4A9FCC37F2C5B1807249FE1285CD4A40B88F');
/*2*/ CREATE COLUMN ENCRYPTION KEY AEColumnKey   
WITH VALUES  
(  
COLUMN_MASTER_KEY = CMK_Auto2,   
ALGORITHM = 'RSA_OAEP',   
ENCRYPTED_VALUE = 0x016E000001630075007200720065006E00740075007300650072002F006D0079002F00620032003700610034006100390066006300630033003700660032006300350062003100380030003700320034003900660065003100320038003500630064003400610034003000620038003800660053FD933BC3E3E6E5FFD935F452A5C4113FF56E4D946D78B22A69415FF8EF69D9B3A5541F2463BBC32D06AC88AE95B4CDBBEE7A9D1DD80043D7C900F28917637F4414565CB3F2B29CEEE5C03DF182C4F62395CDAD59A59BFCBD421889DB9EFB2B5250AA597268011B8ACCFFA7A1B5D846BD476BBD8F8239D2681C800E3BCD848485AEC6E69FE76D06D2E213FB36FCBCA5E8B75FE67D21C1C05EB7CF819AD9F96701116A2B642F690455FC7DC48AEEB1825BB20ECD428F910C002EE3D186706E00F76C608EF78FBB147ABA798309092517A39C9C4031B3857C599B238174AA1E8433A649D63D194278B0A4EFBF15DF4E4B5B4468FB73FC8992B3E34606AB306E2E19BADEE4B38288FF77B9A8E45A56BB321091EF0CF3567076ED27D875286CB2232177F610B9A0DAEBFA34ABA9856A094E26E995987AD050D27954DDB08BED9A34C6D19CBE6B2271A7E716C33850DB8781C9D3B762C0920EED57BB9D2BA581F7AC1A46EA55962200FD26405FE31005D413BA5B624E5AF2770377A13EB68FB681242B8B719499175113E84073013BDC6E03E5F82EC070B9151705F1C564106B93E3C7566E41BAD00209AB4587278640FE225F797DD9BB83284E8A674DFC7F48558441E00BC856161FC93A38E337B050915450E7B0ED848CDB63272B65319B26B45119ED081852DEBE53DFF7A6CD21935FC3CBF2C4852AD01CFF0153B76C196F7667  
);
/*3*/ CREATE TABLE AlwaysEncryptedSampleSQL(
  EmpID INT PRIMARY KEY,
  SSN NVARCHAR(15) 
    COLLATE Latin1_General_BIN2 
    ENCRYPTED WITH (COLUMN_ENCRYPTION_KEY = AEColumnKey, 
    ENCRYPTION_TYPE = Deterministic, 
    ALGORITHM = 'AEAD_AES_256_CBC_HMAC_SHA_256') NOT NULL);

Why do we need two encryption keys?

Yes, Always Encrypted uses two keys – Column encryption key and master encryption key. Column encryption key is used to encrypt the value in the column and database engine stores the column encryption key in the SQL Server instance. However, master key is being stored in external key store like Windows Certificate store or Azure Key Vault and database engine will have metadata pointer to the external key store. Master key is responsible to encrypt the column encryption key.

How do we verify the above implementation?

1. Check the master key

SELECT name KeyName,column_master_key_id KeyID,
  key_store_provider_name KeyStore,
  key_path KeyPath
FROM sys.column_master_keys;

2. Check the column key

SELECT name KeyName,
  column_encryption_key_id KeyID
FROM sys.column_encryption_keys;

3. Check the sys.columns

Select name,collation_name,encryption_type_desc, encryption_algorithm_name,column_encryption_key_id 
From sys.columns where object_id in (object_id('AlwaysEncryptedSample'),object_id('AlwaysEncryptedSampleSQL'))

How do the application encrypt and decrypt the value?

Client application uses Always Encrypted driver. I would suggest you to go through “Using Always Encrypted with the ODBC Driver for SQL Server” to understand better the usage. Anyway, this blog post will be followed up with the next post, on which I am currently working on, to understand the performance impact.

What are the different types of encryption in Always Encryption feature?

Yes, AE comes up with two different type of encryption.

1. Deterministic
As the name suggests, this type will always produce the same encrypted value for a given text. Ideally, this may not be a good option for all the keys as a good intruder can easily understand the value by analyzing data pattern as an example, gender, polar questions etc. As the encrypted value for a given text is always same, the encrypted column can very well part of a join, grouping and indexing.

2. Randomized
As the name suggests, it will produce randomized value which will make the encryption more secure than the earlier.

Gotchas!!!!…..

1. There is no straight forward method to implement AE for existing data apart using the wizard. However, wizard can generate PowerShell Script to do the action later.
2. Encryption method – Deterministic is less secure compared to “randomized”.
3. Encryption method – Randomized cannot be part of joins/groups/indexing.
4. INSERT/UPDATE operations are not allowed directly to table unless through the client driver. We will receive the below error message.
Msg 206, Level 16, State 2, Line 5
Operand type clash: varchar is incompatible with varchar(8000) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK_Auto1', column_encryption_key_database_name = 'test') collation_name = 'SQL_Latin1_General_CP1_CI_AS'

5. By specifying encryption setting in “Additional Connection Parameters” in SQL Connection window, who has access the encrypted table, can see the actual data. However, the login cannot modify or insert new data.

6. Column encryption changes the collation of the string column to Latin1_General_BIN2
7. Encryption will increase the size of the table

See Also

Please refer Transparent Data Encryption

Could not open file E:\TempDB\tempdb.mdf for file number 1. OS error: 32(The process cannot access the file because it is being used by another process.)

Recently we faced an issue that SQL Server was not coming up online in one of our environment.

We found below error messages in Windows application event logs:

Open failed: Could not open file E:\TempDB\tempdb.mdf for file number 1. OS error: 32(The process cannot access the file because it is being used by another process.).
Unable to open the physical file "E:\TempDB\tempdb.mdf". Operating system error 32: "32(The process cannot access the file because it is being used by another process.)".
CREATE DATABASE failed. Some file names listed could not be created.

As we have a clear message, while SQL Service starts, it was not able to create the TEMPDB as the mdf file is being used by another process.
When it happens, it will not start the SQL Service even we manually start it through services.msc. However, we identified, the service will come online once we restart the server.

We started our investigation verifying whether more than one instances are pointing to the same directory for TEMPDB files. Though we have multiple instances on the environment, we made it sure those instances are configured separately and properly. Once we confirmed the SQL Server configuration, we took our attention to other services especially with AntiVirus services. We found that there is Mcafee software running, but unfortunately, we could not find any reference on ProcessExplorer as well. However, we decided to stop the McAfee and see if the issue is related to the AnitVirus scan. Now, it is almost more than two weeks and we did not experience the issue.

Takeaway, if you find similar issue, Instead of disabling or stopping the services related Antivirus, we can exclude certain files associated with SQL Server from the scan.

Here are few directories that can be excluded from anti-virus scanning tool:

1. C:\Windows\System32\LogFiles
2. C:\Windows\System64\LogFiles
3. C:\windows\System32\GroupPolicy\
4. C:\windows\temp
5. C:\windows\SoftwareDistribution\Datastore
6. C:\windows\SoftwareDistribution\Datastore\Logs
7. C:\\Temporary ASP.NET Files
8. C:\Users\\AppData\Local\Temp
9. C:\Users\Default\AppData\Local\Temp
10. C:\ProgramData\Microsoft\SQL Server
11. C:\Program Files\Microsoft SQL Server
12. Exclude SQL Server Data Files (*.MDF, *.LDF, *.NDF)
13. Exclude SQL Server Backup Files (*.TRN, *.BAK)
14. Exclude Full-Text Catalog Files (*.SLS)
15. Exclude sqlmangr.exe
16. Exclude sqlservr.exe

Please give your feedback if this post helps you!!!

Memory optimized Temporary table in SQL Server

Today, we will quickly see how do we create memory optimized temporary table in SQL Server and how to use it efficiently.

I would recommend to read previous blogs before we continue further:
In-Memory OLTP and Memory Optimized data filegroup in SQL Server 2014
In-Memory OLTP: Compatibility Check for Datatypes in SQL Server
In-Memory OLTP: Compatibility Check for SQL Server and Database

Before we start the technical aspect, let us be clear why do we need memory optimized temp tables in SQL Server. Temp tables are very powerful objects in SQL server which is created in TEMPDB. These tables are, as name mentioned, created for a short live may be a for a batch scope, procedure scope etc. With in the scope, we can use the temp tables multiple times. So, many developers would use these objects to populate the data as required for thier operations and use these objects instead of querying the original physical table. Not to wonder, now days, the usage of temp tables are more and many is thinking its a free tool 🙂 .

But, on flip side, as we create huge number of tables, there is also a performance imapct in the system. There is only one database “TempDB” per instance where SQL Server would need to create these temp objects and do the oprations. So, it is observed there are some serious performance issues with tempDB at times. I am not covering tempdb performance issuesin this blog, but introduce memory optimized temp tables introduced in SQL Server 2014 which we can efficiently replace with normal temp objects to avoid performance issues.

Lets get into Techinical now –

Here is a sample example of creating a physical table and a procedure.


Create Table T1 (PeopleID int Primary key, CourseID int)

Insert into T1 values(1,100),(2,100),(3,200),(4,200),(5,200),(6,300)
GO
Create procedure usp_Normalproc @CourseID int
As
Begin
	
	Create Table #TempTable(PeopleID int, CourseID int)

	Insert into #TempTable (PeopleID,CourseID)
		Select PeopleID,CourseID From T1 where CourseID=@CourseID
	
	Select * From #TempTable
End

Exec usp_Normalproc 100

Now, lets look at how we convert this temp table usage to memory optimized temp object in the procedure.

First, Lets create a memory optimized table as below:


Create Table TempTable(PeopleID int, CourseID int,
SpidFilter  SMALLINT    NOT NULL   DEFAULT (@@spid),  
INDEX ix_SpidFiler NONCLUSTERED (SpidFilter),  
 CONSTRAINT CHK_TempTable_SpidFilter  CHECK ( SpidFilter = @@spid ),)
 WITH  
        (MEMORY_OPTIMIZED = ON,  
            DURABILITY = SCHEMA_ONLY);

SpidFilter – is a new column introduced as part of conversion to identify the rows for the session created. This has been indexed for performance.
CHK_TempTable_SpidFilter – This a constraint to make sure the spid has been populated

As next step, We need to create a function as below which is used to filter data for session.


CREATE FUNCTION dbo.fn_SpidFilter(@SpidFilter smallint)  
    RETURNS TABLE  
    WITH SCHEMABINDING , NATIVE_COMPILATION  
AS  
    RETURN  
        SELECT 1 AS fn_SpidFilter  
            WHERE @SpidFilter = @@spid; 

Finally, we will create a security policy to filter the records from the temptable as per the session id its being used.


CREATE SECURITY POLICY dbo.TempTable_SpidFilter_Policy  
    ADD FILTER PREDICATE dbo.fn_SpidFilter(SpidFilter)  
    ON dbo.TempTable  
    WITH (STATE = ON);  

Once we set up memory optimzed table and filters, its time to change the procedure as below:


Alter procedure usp_MO_proc @CourseID int
As
Begin
	
	--Create Table #TempTable(PeopleID int, CourseID int)

	Insert into TempTable (PeopleID,CourseID)
		Select PeopleID,CourseID From T1 where CourseID=@CourseID
	
	Select * From TempTable

	Delete From TempTable Where SpidFilter = @@spid
End

It is very important to delete the data from the memory optimized table. On failing, this will create multiple records into memory optimized table if we re-run the procedure within the same session(please note that its very likely during testing).

Hope you enjoyed this blog, looking for your valuable feedback on the same.

SQL Server detected a logical consistency-based I/O error: incorrect checksum

Recently, we encountered an issue with a restored database stating the below error message:

Msg 824, Level 24, State 2, Line 28
SQL Server detected a logical consistency-based I/O error: incorrect checksum (expected: 0xafbb455a; actual: 0xaf9a455a).
It occurred during a read of page (1:310732) in database ID 161 at offset 0x00000097b98000 in file ‘F:\DATA1\database.MDF’.
Additional messages in the SQL Server error log or system event log may provide more detail.
This is a severe error condition that threatens database integrity and must be corrected immediately.
Complete a full database consistency check (DBCC CHECKDB).
This error can be caused by many factors; for more information, see SQL Server Books Online.

As the error message is pretty straight forward, a clear issue with some consistency (I/O error). And this has been even confirmed by running a DBCC CHECKDB. To resolve the issue, we decided to go with REPAIR_ALLOW_DATA_LOSS option as this is a lower environment issue. I personally do not recommend to use this option for production databases as there is a chance of data loss.


alter database <database>  SET single_user with rollback immediate 
dbcc checkdb('<database>',REPAIR_ALLOW_DATA_LOSS)
alter database <database>  SET multi_user with rollback immediate 

Before we do this option, just curious to understand the object being corrupted from the error message.

Using DM_DB_DATABASE_PAGE_ALLOCATIONS

The below code will give you information on the objects. Please note the below is supported only from SQL 2012 and later versions. For earlier version, you need to use DBCC PAGE commands.


SELECT DB_NAME(susp.database_id) DatabaseName,
OBJECT_SCHEMA_NAME(ind.object_id, ind.database_id) ObjectSchemaName,
OBJECT_NAME(ind.object_id, ind.database_id) ObjectName
FROM msdb.dbo.suspect_pages susp
CROSS APPLY SYS.DM_DB_DATABASE_PAGE_ALLOCATIONS(susp.database_id,null,null,null,null) ind
WHERE allocated_page_file_id = susp.file_id
AND allocated_page_page_id = susp.page_id

Later, we identified the issue is associated with backup and we took a fresh backup of the database and restored. It worked like a treat!!!

How to remove space in a column names of table in SQL Server

Here is a script to rename the column names to avoid the space(s) in the column name.


create table test_table([column test space] int)

select * From test_table

--Script to do the change in column name

Declare @SQL nvarchar(MAX)=''
Set @SQL =(
select ';EXEC sp_rename '''+ B.name+'.'+a.name +''', '''
+replace(a.name,' ','') +''', ''COLUMN'''  
from sys.columns A
Inner join sys.tables B on A.object_id = B.object_id and  
OBJECTPROPERTY(b.object_id, N'IsUserTable') = 1
where 
 system_type_id in 
(select system_type_id From sys.types ) and charindex(' ',a.name)!=0
FOR XML path(''))

print @SQL
/*exec( @SQL) This has been commented, verify the results and execute*/
--check the column names
select * From test_table
--Cleanup the table
Drop table test_table

ALTER DATABASE failed because a lock could not be placed on database ‘dbname’. Try again later.

Problem statement

As a DBA, you will be facing the error message “ALTER DATABASE failed because a lock could not be placed on database ‘dbname’. Try again later.” often for certain actions like taking database offline/setting single user etc.

Analysis

This is due to other processes locking your current operations.

Resolution

1. Find out the query which is blocking the current operation using the below query
Live monitoring query

2. Kill the process and take the database in SINGLE_USER

KILL 191 --session id from the first query related to the database
USE MASTER
ALTER DATABASE DB905 SET SINGLE_USER WITH ROLLBACK IMMEDIATE

3. Take the database offline

ALTER DATABASE db905 SET OFFLINE WITH ROLLBACK IMMEDIATE