Finding most used alerts and monitors to override

I had a requirement from one of the teams that uses SCOM monitoring – the requirement was to change alerting level on scom alerts from critical to warning for all servers that were not on production for BizTalk management pack. Now there are several rules and monitors related to this management pack – or in general any one management pack – there is no quick and easy way to override all of them. To do so manually would be very cumbersome and time consuming. Instead – a better way to approach this would be to find rules and monitors that generate the most alerts and override them instead. This would create the override for most relevant rules and monitors – the others can be overridden as and when they happen. The query I used to find most used rules and monitors is as below. I wanted the BizTalk management pack – this can be substituted with any other as necessary.

 

Use OperationsManagerDW

SELECT

alt.AlertName,

COUNT(*) AS AlertRepeated

FROM Alert.vAlertResolutionState AS ars INNER JOIN

Alert.vAlertDetail AS adt ON ars.AlertGuid=adt.AlertGuid INNER JOIN

Alert.vAlert AS alt ON ars.AlertGuid=alt.AlertGuid INNER JOIN

vManagedEntity ON alt.ManagedEntityRowId=vManagedEntity.ManagedEntityRowId

WHERE

alt.AlertName like ‘%biztalk%’

GROUP BY alt.AlertName

ORDER BY count(*) desc

TSQL Tuesday 66 : Monitoring

This TSQL Tuesday post hosted by Cathrine Wilhelmson has to do with Monitoring.

During my many years as a DBA i have had a fair share of experience with monitoring tools – some of them as listed:

1 Spotlight for SQL Server – great tool for server level monitoring with very cool graphics. What i really liked about spotlight was that it was so very non-techie friendly – so whenever the director or VP chose to walk by and wants a peek into server health – all I needed to do was to pull up the screen and show him the dials whizzing around. On the flip side, spotlight did not help a whole lot with query tuning. It was also rather difficult to integrate with our ticketing system to generate tickets.

2 More recently, I have been working with Systems Center Operations Manager as our enterprise wide monitoring system. SCOM is a really complicated monitoring system to work with – but once you get familiar with it there is a lot it can do – in particular what I really liked was its ability to open tickets on its own within our ticketing system(with some customization of course), as well as close alerts on its own too when the alerting situation was resolved. SCOM calls for a dedicated admin  (which is part of my job), and a lot of tuning to get it down to where it only alerts on what you need without a whole lot of extra noise. SCOM is also NOT a tuning tool – it is only an alerting tool.

4 There are some situations that fall completely outside the realm of third party monitoring tools- for example, we need an alert whenever a user is added as a server role on some servers. We had to write a really simple script that would pull people who are on server roles, set it up as a job to email us every day in case something changed.

**Beginning of script – authored by Mala Mahadevan

SELECT role.name AS Name, member.name AS MemberName,
CASE WHEN (role.is_disabled) = 1 THEN ‘DISABLED’ WHEN (role.is_disabled) = 0 THEN ‘ENABLED’ END as     ‘Status’

INTO #temp_dbo
FROM sys.server_role_members

JOIN sys.server_principals AS role

ON sys.server_role_members.role_principal_id = role.principal_id

JOIN sys.server_principals AS member

ON sys.server_role_members.member_principal_id = member.principal_id AND

–filtering for logins that come from our domain
member.name LIKE ‘prod\%’

–Format email

SET @tableHTML =    N'<H1>Server Role members on Server A</H1>’ +

N'<table border=”1″>’ +

N'<tr><th>Name</th><th>Role</th><th>Status</th>’

+

CAST ((

SELECT  td = CAST(([Name]) AS nvarchar(100)),”,

td = CAST(([Membername]) AS nvarchar(100)),”,

td = CAST(([Status]) AS nvarchar(100)),”

FROM #temp_DBO

ORDER BY name

FOR XML PATH(‘tr’), TYPE) AS NVARCHAR(MAX) ) +    N'</table>’
SET @Results = isnull(@tableHTML, ”)

–Email the team on findings
EXEC msdb.dbo.sp_send_dbmail

@recipients =

‘mydbateam@mydba.com’,

@subject =

‘Server Roles on Server A’,    @importance = ‘Normal’,    @body =

@Results,    @body_format = ‘HTML’

 

DROP TABLE #temp_DBO

What it took to stump the optimizer

Sometime ago I was working with a Microsoft support technician on a ticket related to some database issues on SCOM. The technician wanted to run a stored procedure to clear out some of the backlog we were seeing with a batch job.

When he ran the procedure (via Query Analyser) – we encountered an error as below. Basically an error that is wee bit rare, the optimizer telling us that it is stumped and the query is too big for it to process. The technician moved on to other ways of resolving the issue as fixing this problem involved touching some procedures that came with the product and was beyond the scope of our call. But after we were done, I was intrigued to find out what caused the optimizer to give up.scomoptimiser

I went to Line 177 of the specified stored procedure and found a statement that looked pretty small on the outside:

DELETE dbo.[CategoryOverride]

FROM dbo.[CategoryOverride] CO

JOIN #BaseManagedEntitiesToPurge BMETP

ON CO.[InstanceContext] = BMETP.[BaseManagedEntityId]

The table CategoryOverride had about 300 rows..and  on checking what went into the temp table – it wasn’t all that large either, only around 100 rows. Looking at the estimated plan from the optimizer gave me nothing – a perfectly ordinary plan with an equijoin on a clustered index. So what was causing the problem?

I looked to see if the table CategoryOverride had any triggers on it. None. Then I went on to looking at keys. The table had one foreign key defined on it.

scomo1

I decided to script this key and see what it showed me.

ALTER TABLE [dbo].[CategoryOverride] WITH CHECK ADD CONSTRAINT [FK_CategoryOverride_ManagementPack] FOREIGN KEY([ManagementPackId])

REFERENCES [dbo].[ManagementPack] ([ManagementPackId])

ON DELETE CASCADE

GO

There was my culprit – an ON DELETE CASCADE. So many other tables is this touching? A quick look at the dependancies on this table showed me no less than 14. For one record to be deleted – there were dependancies on 14 tables. I picked one record on the main table and counted how many dependant records were there to clean up. It was about 6000. So the delete statement was far from a benign straightforward delete – it had a huge amount of data to clean up and the optimizer could not generate a plan across 14 tables some of them having more cascade deletes on them.

Moral of the story – never assume the simplicity of a statement based on looking at it, or what the query optimizer tells you. The truth may be much deeper. If I had the choice of fixing it I would remove the cascade delete and go on to cleaning up data step-by-step in each of the dependant tables.

TSQL Tuesday #61 – Giving back

This TSQL Tuesday is hosted by my friend Wayne Sheffield. The topic is on ‘giving back’..what do you think you can ‘give back’ to the sql community in the upcoming year?

When I was new to this country – I had a friend who was a psychotherapist by profession. Towards the holidays – when everyone took time off, went shopping, decorated homes and so on – she would be working double , sometimes triple shifts. She worked for herself, so nobody really made her do this. But it was the time when business boomed for her. More people were depressed and low towards the holidays causing more therapists to be in business. Now, I come from a country with a lot of poor people. Seeing more people depressed instead of happy during a festive season was news to me – but that is topic for another post. The point as she explained to me was that the need to give without adequate self nourishment made a lot of people feel very depleted and as a result, depressed and low. In some ways , our small community is also reflective of the bigger outside culture – we expect people to give more and more – volunteers need to do more, the people on PASS board need to do this and that, MVPs and friends who are doing well need to recommend jobs…on and on. What are we doing to ensure appreciation and self nourishment within our community?

I don’t mean prepping people up or tweeting about how much we love each other and so on , although those things are important. I mean things like looking out for someone who is out of work. If you find any opportunities that he/she may be interested – let them know. Put in a word of recommendation, if you can do it. Share your own stories, not just of successes, but of failures. Like the time when a technical interview was really hard and you knew in 10 minutes that you were a poor fit. Or the time when you wondered how having 500+ linkedin contacts is not leading to any successful job leads. Or when everyone is talking of the sqlserver job market booming and there is nothing remarkable happening in your neck-of-the-woods. We all have times like that. Sometimes, sharing those stories is the best thing we can do to help someone who is depleted. As I write this, I know of 4 people who are looking for work. They are all smart, hardworking, caring individuals who have given to community and are in need of self nourishment. My goal is to be there for them in whatever way I can. And for people like them, in the year to come.

User Group Funding: Twitter Chat summary

I was part of a very interesting chat on twitter on how to find funding for user groups and SQL Saturdays. The conversation was initiated by Brent Ozar with an RT of Andy Warren’s blog post stating that running chapters is a lot of hard work. It was followed up with an active discussion on funding and how to find more funding to support chapters, particularly smaller ones. Those who participated include – Brent Ozar, Grant Fritchey, Kendal Van Dyke and Andy Warren. Matt Velic and me added our thoughts also. Following are some interesting observations.

On funding for small groups:

Me: ‘Funding for small groups has become inconsistent after UGSS and Idera pulled out of consistently sponsoring.’
Kendal: ‘Ideally that’s where SQLSaturday activities can help fund the group for the year.’
Matt: ‘SQL Sat would have to charge a fair amount to fund for a whole year’.
Brent: ‘Charge $25 for SQLSaturday,still the deal of the century’.
Andy:’Hard to justify cost/effort/reward for small groups if you’re a sponsor. Have to find ways to change that’.
Kendal: ‘Having a marketing plan, good look/feel, consistent messaging – all help bring in more sponsors.’
Grant:’ Fact is, small UGs suffer. I know. Trick is, minimize your needs, don’t emulate big groups.’

On topics and speakers:
The topic deviated to if or finding big-name speakers was important or as important as topics. I spoke to my experience that big names draw big crowds – at user groups or sql Saturdays. Others chimed in as below.

Brent:There’s less of a “celebrity” factor in the SQL community than folks think.For most attendees, local presenters *are* stars.
Andy: ‘Topic matters as much, or more.’
Grant : ‘ Another vote for more. Topic wins huge. I’m seeing that more & more.’
Grant: ”Fight like heck to get big name speakers, even if it’s just remote.’

On drawing bigger crowds of people :
We had some debates on quality versus quantity of people. Charging a fee might mean fewer people but draw those who are really interested.
Brent – ‘Vendors want quality too, not just quantity’.
Grant – ‘Speaking as a vendor, we want quality, but let’s face it, quantity has a quality of it’s own.’

Everyone agreed that Andy had done a great job with Orlando SQL Saturday and also with blogging consistently on these issues. We look forward to more posts and guidance from him (with other thoughts and ideas also). as we move forward into the next year.

 

 

TSQL Tuesday #60 – Something new learned..

This TSQL Tuesday is hosted by my good friend Chris Yates – the topic is on something new learned in the recent past. It is a simple topic but can sometimes be hard to articulate. I am a big believer in an open mind and learning anything new that comes my way – and have learned that the older you grow the harder this can be. The one practical lesson I claim to have learned, finally, in the recent past – is to sell myself and my work better. Or..to put it in other words, that publicity for what you learn and do is as important as the work itself.

I grew up in a culture that did not think too well of people – in particular women, being very extroverted. That combined with the fact that I was home-schooled for health reasons did not make me a huge extrovert by nature. My good friend Kevin Kline once did a survey of geeks and explained that most people who take to computers in a big way are, in some ways introverted. I don’t think being introverted has anything to do with self esteem, or even being shy or anti social. I just think these are preferences around how we are – that we prefer quality company over numbers, and are somewhat deliberate in our thoughts and actions. I do think though that it does impact how we sell ourselves – or publicise our work. For someone to know you you need to get out there and show them who you are – or it is more likely that those people who know far less than you do and drum up their work better will get the right chances to get ahead.

In my 13 years of attending the PASS Summit – I have always tried to spend time in classes, attending the occasional evening party. This time, I did something different. I spent most of my time networking – hanging out in community zone, asking people if they’d like to do lunches or dinners where we shared each other’s cares and concerns, and introducing myself to many SQL celebrities I had only known by name. I found the experience very rewarding – I got a lot of insight into options for career advancement, upcoming trends and changes in nature of work (more telecommute, more data analytics, more ‘unlimited’ vacation…) – not to mention the emotional support from a  huge number of #sqlhugs. I was selective, of course – in some ways, like I have always been. I usually hang with people whom I feel are genuine at some level, not just all about fluff, and are intuitively easy to relate to. That is something that I will want to keep.

My good friend Grant Fritchey once remarked  to someone who was a very strong techie and yet very humble and unassuming – ‘you need to be more assuming’ (he meant that you need to sell yourself more/better). That , in one sentence has been what I learnt recently – to be more ‘assuming’ while being true and genuine, at the same time.

PASS Summit 2014 – People to meet

I am late writing this…between getting ready to take one week away from work, things to pack and keep the home in a reasonably tidy shape there was no time to blog. But I have been gathering thoughts in my head as I went by doing my chores – on whom to meet and what to do. This is my summit #14 in a row, and have never ever regretted coming back here because of the friends I make and the connections that enrich me in every way.

1 Whom to meet – I am one of those many people who learnt the hard way that the community that matters the most are those in your own neck-of-the-woods – people who work at companies nearest to you, volunteer at your sql Saturdays and user groups and so on. So my very first preference is my breakfast with the team from Louisville – I know several of them already and most do not have a blogging or tweet presence. But I do look forward to meeting Dave Fackler – the user group lead after me who has been doing an awesome job keeping up the meetings, and Chris Yates – whom I met on twitter and is growing to be a good member of our local #sqlfamily. Our group is scheduled to meet on Wednesday morning for breakfast at the summit.

2 The non celebrity friends network – I like to call it that – we are people who are regular attendees of the conference – none of us are celebrities/MCMs/MVPs. Some are ardent PASS volunteers . We like to support and offer grounding to each other as we grow in our lives and careers. My long time friend from St Louis user group Julie Bloominquest, SQLSaturday champion Pam Shaw, Microsoftie friend Karthika Raman, other friends like  Meher Malakapalli, Dan Brennan and Adam Belebczuk.

3 New people to add to the network – Some people I have only known via twitter or other people’s blogs – Doug LaneNancy Daniels , Theresa Iserman, Andy Galbraith, Joey D Antoni are some of them. I think I have more just not getting names out yet.

The number of people I have known over the years and want to catch up with is simply too long for a blog post…that is what makes the summit worth returning to every single year.

Some posts you may want to read that are similar are from two awesome #sqlgals – Tamera Clark and Mickey Stuewe.

http://mickeystuewe.com/2014/10/29/who-you-gonna-meet/
http://clarkcreations.net/blog/last-minute-thoughts-for-summit-2014

 

Wishing everyone a happy conference!!

 

 

Previous Older Entries

Follow

Get every new post delivered to your Inbox.