Monday, December 07, 2009

Is Windows Mobile Relevant?

There is an interesting read on BetaNews about the Windows Mobile platform. Before I get flamed for being a Apple junkie, I'll say right now I do not have nor want an iPhone. The closed platform and interface do nothing for what I need.

My current cellphone has been trending downhill for the past few months. It's an HTC Touch Pro device with WM 6.1, and I hacked it to run WM6.5 that I downloaded at xda-developers. Those great hobbyists do a wonderful job at packaging roms which unlock the full value of phones. I've tried a few different roms but the "Energy rom" line was the most stable and best one I've seen.

Now I'm sure you're thinking "the phone worked fine until you hacked it". This is the opposite, the phone out of the box from Sprint was buggy, laggy, and was a fail at basic telephony. Not to mention the habit of draining the battery before lunchtime. By researching and installing a custom rom I was able to get another 6 months of use from the device, else I'm certain it would have ended smashed into a million pieces on a sidewalk somewhere.

This leads us to today - WM is coming out with WM 7 next year. This is supposed to be an awesome system. WM 6.5 excels where Apple fails. It's open development, lots of good resources but on the flip side a lot of the usage is wonky and seems 2003-ish. I cannot wait for a better system in a year from now when I need a smartphone today.

I purchased a HTC Hero last week running Android. Android is made by Google and is quickly gaining market share among smart devices. It didn't have a dedicated keyboard so I thought I would be bothered by the virtual keyboard, but so far it's not too bad. The phone works well and it seems that the flow and use of the software makes sense. It's much more user intuitive than the Microsoft platform as it exists today, but I'll wait for the newness to wear off before I comment on the overall package. If Microsoft is destined to have a presence in the mobile market, WM 7 must be a compelling "wow".

Sunday, December 06, 2009

14 Years and 1.2 Million Miles Later

Last night I had the great pleasure of flying on Southwest airlines into Kansas City. I haven't flown on Southwest much this year but when I had this opportunity I jumped on it. Boarding the flight was painless and I picked an aisle seat right over the wing to try and have a similar experience as before. We arrived at KC Airport at gate 31, and when I departed into the terminal, I had a strong feeling of deja-vu.

It was June 1995. I want to say it was the 8th day of the month, but I'm not sure. A family member had purchased me a ticket to fly from KC-Chicago on Southwest. I had never flown on a commercial aircraft before and was a little nervous. I won't state my age or give away too much information, but I'll say I was "young". I left my car at my uncle's home and they took me to the airport, I checked in my bag, and waited for the flight. At that time Southwest used those old plastic number cards, and I believe I had a 'B' card. The plane was this huge light brown thing, later I learned it was a 737-200. We boarded the flight through gate 31 and I found a seat right behind the wing on the left side of the aircraft. I recall that flight had some seats that faced to the back, and there was a very attractive blonde girl in one of those seats who ordered a beer on the flight. I'm sure I had a Coke. My first feeling was the massive acceleration on take off, and the rest of the flight was an awesome experience for a young boy.

One of the neat things about consulting and building a business is the opportunity to travel to distant locales. Some people enjoy traveling, some see it as part of the job, and some hate it. I understand all these perspectives. Last night as I recollected on my experience, I thought of the many airports I've visited since that day in June 1995. I thought of the planes I've flown on that would make a 737 look like a tiny toy. I thought about the many people I've met along the journey, both attractive ladies and nice guys. I then thought about what I know now vs. what I knew in 1995. It helped me to remember that the world is full of opportunities, that we all have a chance to make what we want, to live our dreams. I am living mine, talking, practicing, and helping companies become more efficient through the use of information technology and business intelligence. In June 1995 I was working at Walmart, thinking of how we (myself and the four other guys I worked with) could get bleach loaded onto shelves faster, work the stock faster, or sell more items through use of displays. Now I am thinking of how I can use IT to help a client complete financial audits. Wow, what a 14 years it's been. I'm certain that I will look back in another 14 years and saw that in 2009 I had no idea how great it would all become.

Friday, October 30, 2009

Update on the Data Modeling Tool

One of our clients was testing Toad Data Modeler to see if it would suffice as a replacement for CA ERWin to construct a dimensional model - the verdict is in, and it's "No." Toad Data Modeler does regular ER diagrams not not a dimensional model. Looks like CA ERWin is going to be the tool of choice for this client as well as many of our other clients.

Speaking of Toad for SQL Server, I'm not convinced there is a need for the product as SSMS is such a great tool. Microsoft really is sitting in a sweet spot right now by providing SSMS with the SQL Server license. Toad for Oracle is a necessity with the Oracle products.

Monday, October 26, 2009

It's a Great time to be in BI

Yes it is, a great time for BI. Why is this? BI saves customers money, customers spend a little money in a BI application or dashboard but save a lot more with intelligent data and all the good things that come with it.

We just brought on a great person to fill the Marketing Services Consultant position at Durable Impact. Stay tuned for a blog introduction of this person and keep your eyes posted, we're soon going to be looking for at least five more SQL/BI Developers. Contact us on our webpage if you want to be notified when the positions open. It's always helpful to include a short paragraph or two about yourself and what interests you, so we have a good relationship from the beginning.

Saturday, October 17, 2009

Website Moderators

Thanks to everyone who attended my presentation today at SQL Saturday. It was a great crowd and I enjoyed it immensely.

I am looking for one or possibly two website administrators. The position entails spending 1-2 hours per week working thedamndata forums. Your responsibilities will be to approve new users and posts, create postings, and remove inappropriate content. Again, this should only take one or two hours per week. The DI staff is doing it right now but it's keeping us busy dealing with spam bots, porn operators, and illegal prescription peddlers who continually wreck havoc on the website.

This is an unpaid position, but as compensation for your time, you will receive a one-time $100 gift card to the retailer of your choice. We may provide other perks from time to time, but the primary reason we want someone is for their passion to speak and write about data and spend a few minutes assisting the website as required.

Please email me using the contact us form at www.durableimpact.com if you are interested.

Friday, October 16, 2009

Data Modeling Tools

Most software apps have a very strong following and drive strong opinions from the users, taking for instance Windows vs. Mac. Those who use Windows often don't understand Mac, and those who use Mac despise Windows. Data modeling is one tool where I haven't found this type of approach, almost everyone I've talked to either uses ERWin or has heard good things about it.

I was working with a client this week who needs to get a data modeling tool. I've recently been working with some clients who have their own version of data modeling using Microsoft Visio and powerpoint slides, but this really isn't a good substitute for a true dimensional modeling tool. I've used ERWin in the past at a few clients and it's a great tool, allows one to reverse engineer the database, or better, to create a new model, creating the DDL for the model, and then versioning it for source control. The one problem with ERWin is that is isn't cheap, thus many look to Visio. Visio is a good tool in it's own right, but it's not a true data model tool yet. I suspect this is the direction that Microsoft is trying to go in and maybe with Visio 2010 it will meet that goal.

One tool I was made aware of is Toad Data Modeler. Toad products are made by Quest Software, and I've been working with Toad for Oracle since the beginning. I'm very pleased with the Toad database products, and have already recommended to this client that they buy Toad for DB2, which they have done. I downloaded a version of Toad Data Modeler and my first take of the tool is that it's going to work great for what we need to do, and it's at a very attractive price point as well.

I always stress to clients that they have a printed data model, typically where the developers can see it and visualize the relations between the tables in the model.

Tuesday, October 06, 2009

Sales and Making Sales

My friend Andy had a blog posting last week about sales guys in the software field. Selling is always an interesting part of running a business and most likely the part that causes the most headaches, but without sales there is nothing else.

I'm typically distrustful of most commission-based sales where I'm the consumer. This is true for auto sales or buying suits. I bought a suit last week at a major mall-based retailer and I asked the guy (after purchasing, bad on my part) if it was quality. He went to say something and then stopped, smiling and saying "everything we sell is quality". It's blanket statements like that that make me second-guess my purchase.

I've stopped attending many industry events and meetings because they have turned into sales affairs. I attended one industry event where, of the approximately 60 attendees, at least 30 were sales guys hunting for leads and 20 were unemployed guys looking for work. The other 10 were people like me just trying to learn more about the product and maybe meeting a person or two in the process. I also wrote back in April about getting kicked out of an Oracle event that was a "briefing" but truly was a sales pitch. I've found a much different experience at true community events (like SQL Saturday), and while there is always the element of selling involved, it's not excessive. I have received a few unwanted emails from being on mailing lists coming out of these events, but that's more the exception than the norm. I hope that as SQL Saturday events grow they continue to stay true to their roots.

This brings me to a conversation I had with a client recently. Selling consulting services is kind of like dating, sometimes it's smooth and sometimes it's not, sometimes there is a return phone call and sometime there is not. One of my major disappointments when first getting into this business is the amount of sales leads that seem very hot and then turn cold. It does come with the territory and one has to learn not to take it personally. Anyways, the client I was talking to described to me his process of choosing a company to perform the work. He spoke about other firms which had presented and how things had went with them. What it came down to, in his mind, was trust. I'm pretty plain-spoken which has it's strengths and weaknesses, but it worked in this case that this client believed what we (my partners and I) said, and our straight talk gave them a great sense of comfort. I can relate to that, as I stated earlier, I'm pretty reluctant when listening to sales pitches and always trying to understand all the angles of the discussion. It was pleasing to hear that at the end of the day, the time-honored traditions of honesty and integrity were able to overcome bigger names and slicker salesmen.



Saturday, October 03, 2009

A Bumpy Flight

What was your first thought when you read the title "A bumpy flight"? Were you confused, trying to figure out if I was talking about the economy? Or did you think I was talking (as I do) in analogies?

I was talking about both, and a true bumpy flight.

In reality, I just returned from Dallas on this flight. The 2 hour flight was quite bumpy the entire way, so much that the flight attendants were told to sit down multiple times and barely completed the meal services before landing. I find myself to be a fickle flier and the slightest bumps make me a little nervous, not really because of safety concerns, but I always think the next round of bumps will lead to one of those massive bumps we see on CNN causing multiple injuries. Thankfully we didn't twist a tail and I'm here to tell you about the time this past week that I used the "bumpy flight" analogy.

We just started working with a large client on a data warehouse project. I will talk more about that later, but suffice it to say most would have heard of the name. A partner and I spent the past week working with the client on data discovery and trying to get them set up for the project. A critical part of the project is setting appropriate expectations. This company has never had a data warehouse before, so it's a "greenfield" development project. This is the best kind, from my opinion, because we get to do things right the first time. A majority of our projects to date have involved enhancing or repairing someone else's work so this is a welcome change as a different kind of challenge. Part of my duties this week were to work with senior management (CFO and Director level) to get an idea of what they wanted from the project and try to answer any questions they had about the process. I really enjoy working with management and listening to their concerns, that is true consulting at it's best.

I had a great meeting with the chief product engineer. Part of answering the business questions requires a deep understanding of the business. This person had a lot of great comments and thoughts and our scheduled thirty minute meeting quickly turned into a great two hour discussion. Being a good engineer, he was very interested in risk factors for the project and wanted to know what risk mitigation plans I had prepared. It's true that starting any project risks missing a key requirement and the best way to mitigate that risk is to keep the business users as closely involved with the process as possible. He was satisfied with that answer and as a follow up comment I used an analogy that brings us to the title of this posting. I explained to him that a new data warehouse project will have it's growing pains and to expect them, I told them that "it's similar to a flight from Dallas to Tampa where it's bumpy the whole time. There will be times that we all wonder why the air isn't smooth and there will be bumps along the way but the safety of the aircraft is never in question and we have complete confidence we will arrive safely at our destination."

I love analogies, especially when they come true.

Tuesday, June 09, 2009

SQL Server DBA / Reporting Services

Hello all, I'm going to keep this short today.     DI is needing to grow our stable of consultants, and specifically I'm looking for a couple positions with the following:
  - SQL Server DBA experience
       - Experience with 2005 and/or 2008
       - Prefer 2005 to 2008 upgrade experience
  - SQL Server Reporting Services experience
       - Reports in 2005 and/or 2008
       - Using analysis services for cubes

We will sponsor your MCDBA or other Microsoft certifications.     The first test is to see if you can figure out the email address below!  

Please email me at: resumes at durable impact dot com so we can start a discussion.




Monday, June 01, 2009

Employee vs. Consultant, Topic: Insurance

Today we'll talk about the differences in insurance between working as an employee and working for yourself. Yesterday I explained the difference between contracting and consulting, but both types are considered (by the IRS) to be self-employeed unless they receive a W2, and then you're an employee

Companies began providing insurance as a benefit to attract workers after the shortages in WW2. This benefit has grown into the extreme mess we call "health care" today in the United States. The problem here is that we spend the most on heathcare of any country yet do not receive the best care. Bear in mind, there will be a lot of economics discussion today, but it's necessary to understand the point.

Employers, being the de-facto providers of health care, have a few advantages going for them. The first is adverse selection. Every company will have some people who require recurring care (cancer, diabetes, etc) and some people who require no care, so the insurance pool overall will skew towards the median care requirement being right in the middle. Thus, the portion of insurance for each person will be say, $200, of which the employer picks up $150 and the employee pays $50. (Assuming a single person here). The employee portion of $150 is considered a benefit (and a tax writeoff) to companies.

Let's take the same person under a self-employed scenario. Now, this person knows their own health. We call this moral hazard. They know their favorite foods are cheese fries and buffalo wings. They also know their family history of heart disease. They do not reveal this to the insurance company though when this person purchases insurance for $200 a month, of which the insured is responsible for the entire amount.

There is another self-employeed person who is very fit, has no history of family disease, eats healthy, but balks at paying $200 for heath insurance the person does not think he needs. This is where adverse selection comes into play, as only the sick people or people who believe they will need care buy insurance, and the healthy do not. That is why private cost plans are cost prohibitive. There are some ways around this, as credit unions and other firms try to "pool" people together to buy insurance at discounted rates, to "pull" both of the people (heathly and unhealthy) to buy insurance and average out the risk again. However, this still presents major problems for insurers, who are in the business to make a profit.

Now we enter into the political debate as to whether to force people to buy insurance on the private market, providing the pooling mechanism, or to have the insurance provided by the federal government which again provides pooling. I'm not going to get into this futher except to state that either of those options is preferable to me versus the current option of employers providing insurance.

For those who accept the risks of self-employment and need insurance, there are multiple things you can do. First is to buy a full plan on the open market, which is very cost prohibitive. There is something called a HSA (Health savings account) which when linked with a high-deductible plan, allows people to retain insurance for catastrophic occurences but also has high deductibles to discourage the overuse of the insurance. (If you're interested, this is what I do). If you're married to a partner who has corporate insurance, maybe the best option is to use that person's plan.

If you're going to consider the cost of health insurance in your criteria, factor in the full cost. Employee contibution is $50 every two weeks, and private insurance is $100 a month. Therefore, the premium expense is equal. There are other costs to be factored in here, such as office visit deductibles, but remember that cash is king, and while on corporate insurance I had a prescription once that was a $37 co-pay, but only $21 cash price.

If you're thinking of becoming self-employeed understand that there are options available to you, and with the looming discussion of health-care reform, the options may be more attractive and remove it as a key decision making point in the future.

Sunday, May 31, 2009

Consultant vs. Contractor

Before we go any further in the "Employee vs. Consultant" discussion, I want to clarify a few things. There is a difference between consulting and contracting. Many times the words are used interchangably but this is not correct. I'm certain others will have a differing point of view, but here's mine.

Contracting: hired to do specific tasks, not always a specific range of subject matter expertise needed, for a given duration, very little overview

Consulting: hired specifically for expertise, asked to advise on processes and output, may do some tasks, but work closely with management and employees to instruct and guide

When I first started out I was contracting, but then moved to consulting. Consulting is where you can really make a difference, contracting is more like being a temp employee. Just today I was participating in a conference call with a client, discussing the way a dimensional model is built. One of the attendees asked me why I put both the surrogate key and business key in the fact table, and I replied that I did it in case the surrogate keys were corruputed, to aid in rebuilding them and also for research. Another voice popped up and said that was a good standard for moving forward, and now it looks like that client will be using that process in the future.

Contracting is different though, it's like signing a contract to run a couple thousand test plans against a new piece of software. That requires experience to execute the plans and report results. The difference is that during the requirements gathering and design review, a consultant who had worked with other systems in the industry was brought in to review and provide thoughts for product enhancement. It should be clear that there is a difference in executing a script and advising business process owners and management on features.

There are key differences between contracting and consulting, and it's good to keep them in mind when reading my posts.

Saturday, May 30, 2009

Employee vs. Consultant, Topic: Stability

It's been about a year ago when I wrote my first post about the differences and trade-offs between working for someone and working for yourself. My conclusion then was that you should add up the costs and benefits of everything and then make the best decision. The problem is that the true costs and benefits are obscured and it's hard to make an informed decision. I decided to expand on the topic and have a couple different posts exploring different dimensions of making the best decision for you.

In December 2008 I received a phone call from a friend, the type of call that you never want to receive. He informed me that he had been let go from his corporate job. He's not too far from retirement and almost had his house paid off and was well thought of at his job, but the management had to make a decision that I'm sure was not easy. He was shell-shocked, but quickly recovered and found something with little overall loss in pay and benefits. His statement to me "I thought it would be my last job" was telling. In his mind, he was relatively safe, having traded some dollars in pay (though not much, he was doing quite well) for perceived stability. All of my clients have let people go since last year. I wonder how many of them believed they were safe too. More to the point, I wonder how many believed they were safe and traded a decent amount of salary for that security which they did not receive?

What would you do if you have the choice of two jobs that
- Company A: $78,000 a year, history of mass layoffs and hire/fire shop
- Company B: $70,000 a year, history of never having a layoff
Consider all other benefits as being equal. What would your decision be?

This is a true life scenario of another acquantance of mine. She picked company B, and two years later the company, well known for it's stability (both good and bad) was sold and everyone was laid off. In that two years, she traded $16,000 of salary for perceived stability. After taxes, you have a new Ford Focus. To be fair, there is no guarantee that she would have made two years at company A, but the economy was growing and during that time A did not have any known layoffs.

The point of my exercise, in case you missed it, was to try and remove stability from the equation as much as possible, as stability is a factor that is so far outside of an employees' control. Look at local Tampa company WellCare, a stable company that just did a massive layoff due to some fraud charges. How many DBA or business analysts had any control over that?

Consulting hasn't been too much better over the past year. Many projects have been cut, cutback, delayed, etc and some consultants and contractors have been enjoying time at home. A consultant I know who lost a project was quite concerned and told me he only could survive 2 weeks without a steady gig, if that. Without knowing the details or being judgemental, that's not a good position to be in any way, but in that case being an employee might be a better route as generally layoffs include some kind of package or unemployement, where self-employed does not. (In case you're curious, he went exactly two weeks between projects).

In conclusion, I recommend completely ignoring any discussion or talk about job stability and focusing on other aspects before you make your decision.

Coming up next: Health care, the most dreaded expense.

Friday, May 29, 2009

Managing Servers

Today I want to write about my server management policy and philosophy. This is one cost area that doesn't offer a competitive advantage for us, so it's best to go with the cheapest options and minimize expenses.

My company's policy on server management is that anything production-capable will be managed by someone else. That is, we host everything with a professional provider. We're in the business of consulting and data management, not hardware providers. I don't feel like we can adequately procure and maintain the hardware as well as a company that is dedicated to this task. The inspiration for this decision was David Ricardo, a famous British economist who is most commonly known for the theory of comparative advantage. That theory goes like this: if China is best at making clothes and the United States is best at making computers, then China should make clothes and trade them to the United States for computers. That's a very simple way to describe it, but we all know what we're best at and I want to focus the business on consulting and not hardware management, so we outsource that area.

Hosted servers are expensive on a month-to-month basis but much cheaper in the long run versus buying hardware, paying for a COLO setup, and buying software licenses. I'm certain that the costs will continue to go down with the introduction of Hyper-V on Windows Server 2008. Durable Impact works with both Unix and Windows-based environments, but our hosted servers are running Windows Server 2003. At some point we will move to Windows Server 2008, but only once our hosts do.

Thursday, May 28, 2009

A Trip down Memory Lane

While working at a client site the other day, one of the people I was talking with had a memo from 10/13/1988 hanging up at his desk. This memo concerned how to use the office lighting system (which must not have changed since then), and was yellowed with age, but still readable. I joked with him about the memo and we got into a discussion of the days when "there was one modem for every 10 people". That reminded me of my first PC.....

....to tell the entire story, my parents purchased a Tandy 1000HX system in November 1987. I don't remember much about that computer, but we only had it for a while before my dad traded it in for a Tandy 1000SX. We bought it at a Radio Shack store in their "Tandy Computer Center". It had a 40 MB hard drive, 512 KB of Ram, and 5.25" and 3.5" floppy drives. It ran MS-DOS and my brother and I had a bad habit of just turning it off, which caused the hard drive to crash, thus we got a hard scolding to type C:\PARK to park the heads before shutting it down. We used this computer for years for games and general Word Perfect 5.1 applications and with work and high school I didn't use it for a long time until I got my own computer.

It was a Packard Bell desktop purchased from WalMart for $1399 in 1996. It had a 15" screen and a blazing-fast Pentium 75 with a whole 8 MB of ram and a 640 MB of hard drive. I saved up my own money and purchased a Canon BBJ-4100, one of the first color bubblejet printers for a cool sum of $318 after taxes. I still have both of the receipts, but I recall that printer cost an entire week of work. When I arrived at college shortly thereafter, I discovered that all my dorm mates loved the printer and cartridges rarely lasted a month, and no one could afford to chip in to replace it. I also realized that Packard Bell wasn't so hot, and by summer of 1998 I sold it for $350.

My next machine was a step back, but really a step forward. It was a used 486DX/66 that my dad bought at a garage sale. I loaded it with a cool 20 MB of ram, a 4.3 GB hard drive (which seemed entirely huge at the time), and a decent video card. The only thing it didn't do well was play games but it was remakably snappy for being a 486. Being at the university I had accesss to 10 Base T ethernet and enjoyed the web in high speed.

In spring 1999 I built a new machine using my mothers' credit card (thanks, Mom although I know you weren't happy at the time). It was a Celeron 300A with 128 MB Ram and an actual legal copy of Windows 98. If you remember, the 300A was the unlocked processor that was highly sought after for builders, and I picked one up cheap. I was able to get it to run at 550 Mhz at one point, but I generally kept it at 450 Mhz using my Abit BH7 motherboard. Throw in a 10 GB hard drive and a 17" flat CRT monitor (pretty good size for that time), and I was computing in style.

That computer lasted until 2002 when, after multiple hard drive crashes (now I know why OEM drives were so cheap online), I purchased a Dell Dimension 8200 desktop with Windows XP home edition. It was a P4 2.53 with 256 MB of RAMBUS ram. The computer was a great deal ($750 incl 17" CRT) but only because Dell was trying to liquidate their RAMBUS ram because it was so expensive and DDR had just come out as a competing technology. I paid $200 just to put 512 MB of ram in the computer, but by the time I sold it in 2007 for $350 I had added a 19" LCD (again, when I bought it that was HUGE and rare), a super nice 4xAGP 256 MB Radeon video card, and a 160 GB hard drive. That was the last (and best) desktop I owned and now I'm only using laptops.

In 2003 I picked up a Dell Inspiron 1100 laptop, Celeron 2.0Ghz, 512 MB ram, 40 GB HD, Intel 945 graphics card. It was cheap ($800) but had a 15" screen (1024x768). I could never get used to the screen resolution as everything was too big, and the computer was a beast, but worked well and survived two years of being totally abused in my backpack being trucked to work and back. Somehow I was able to avoid the problems that plauged that lineup (especially the 5100 and 5500) models that used full Pentium chips, somehow the CPU cooling ducts were on the bottom and would suck in dust and clog up baking the CPU. I never had any issues, and the computer worked great until the day I sold it.

In 2005 I picked up a Dell Inspiron 9300 P-M 2.0, Windows XP pro, 1 GB ram, Nvidia 6800 256MB, and 80 GB HD. This is the computer you will still see my using often as it has proven to be a true workhorse. It was one of the first 17" laptops on the market and I'm very thankful to have it, as it's been worth every penny and them some. I just used it today to do a client presentation. The only complaint I have is that the screen resolution is only 1440x900, but starting with the 9300 Dell discontinued the anti-glare HD screen and if you're read this blog before you know my opinions on that matter. I've been watching ebay and I can pick up a matte HD screen for around $200 but it doesn't seem worth it.

I won't even write about the mistake of an HP laptop purchase in 2008, but I also purchased the Apple MacBook Pro 17 in 2008. This is by far the nicest and best spec'ed out machine I've every owned, although it will be hard pressed to beat the Dell 9300 overall. It has a C2D 2.0, 4 GB ram, 320 GB HD, Nvidia 8400 GS 256 MB, and HD matte screen. I'm not crazy about the apple software, accessories, or fashion statement component that came with the purchase, but it's hard to argue with the form and design of the machine. We'll see how long I keep this one, but so far I think it was a good deal overall.

Well, that's about it for my trip down memory lane, now it's time to get back to work. Hope you enjoyed it and think about how you got started, comment on it below. Let's see who has the oldest rig still in daily service. (2005 here)

Wednesday, May 27, 2009

Rates - What is right and what is wrong

The other day I mentioned that the economy was having an effect on the rates consultants can charge for products and services. In most cases, the effect is a negative one. This will be a good topic for today because the rates are always the most important part of a project.

Part of being a consultant and business owner is trying to figure out the right amount to charge clients. At DI, we have a range of rates that we charge depending on the services requested. In general, these rates rise with time as the company grows. I have also seen it go the other way though, as a new technology arrives the first few people can command very high rates for knowledge but then as more competition enters the market the rates are forced down.

I had an interesting exchange with a partner last week. This partner was talking about a potential project for a fairly low rate. The plus side was getting in a project at the ground floor with the potential to develop a long term relationship. The negative side was establishing a benchmark rate that is significantly lower than our existing work and likely severely limiting to any upside we might have with this client in the future. In speaking about this issue with one of my advisors, I called it the "GM" problem. The so-called problem with GM is that the car company is selling discounts instead of cars. When customers think of DI, I would prefer them to think of "quality work" and "excellent value" instead of "cheap". The advisor started laughing and agreed while pointing out that I drive a GM car.

This is a problem that doesn't have a cut-and-dried solution. In my head I have a rough formula that I follow but it's different for every situation.
The key point to remember is that rates are by and large a function of the economy, demand for the skill, supply for the skill, and any other factor (such as brand reputation, client sense of urgency). We can decide what the bottom line is and concentrate on projects over that line, and where your line is begins with you.

Tuesday, May 19, 2009

It's Us vs. the Economy

The other day I received a positive comment about a blog post I wrote last June titled "The Great Debate - Employee vs. Consultant".    The recent events in the economy with layoffs and project cancellations has made it an even more relevant topic today, although some people might not find much choice but to do contracting/consulting due to the lack of full time employee positions available.   To be more specific, there are a lot of jobs available but even more candidates (competitors) for those jobs.  

The past 12 months have shaken even the most die-hard consultants and employees.  For us techies, it hasn't been as bad as 2001, but for those who have lost their jobs, that's not a very comforting thought.  The economy has pressured DI some, but we're all staying busy and producing revenue which is always a plus.  The positive thought is that those who can survive through this recession have a great shot at coming out on the winning side in the recovery.

Part of every recession is a drop in projects (i.e. work) and billable hours and rates.  From my perspective, clients are being much more cautious with their money and scrutinizing hours and expenses much more than the past few years.  Part of DI's value proposition is that we try to always think of the customer, as I did tonight when choosing a dinner entree at the local TGI Fridays, I opted for the wrap instead of the more expensive steak.   That might be a silly example but it's the expenses at the margin that make a huge difference overall.   Our consultants are also taking more time than ever to work directly with client managers to ensure we are providing the maximum value.   Rates clients are willing to pay haven't changed much for us, but we've found some of the competition willing to undercut rates in a major way to drive revenues.  In the end, we do not expect any negative long term effects from this behavior and believe this kind of competition will end later this year.

One major casualty of the economy has been the training market.  Inquiries from training had been a decent part of the business but have become a very minor part.    I expect the training business to be very poor well into the recovery and probably not good until we're nearing full employment, when companies will once again use training as a perk to attract and keep employees.  Right now, people are just thankful to be working and not necessarily concerned with getting professional training.    

At the end of the day, the only thing we can do is to continue moving forward.   In the next couple days I'll be doing an update to the "Employee vs. Consultant" post that is more relevant to the current times, so stay tuned.


Friday, April 24, 2009

My Oracle BI Experience

A colleague of mine invited me to an Oracle event that was held yesterday (Apr 23) in Tampa. There were two events going on - a SOUG Tech Day at the HCC, the noted speaker was Tom Kyte (asktom.oracle.com) and I was looking forward to seeing his presentation. A second event was held concurrently at the Ruths' Chris Steak House in Tampa, titled "Oracle Business Intelligence Briefing". I agreed to meet my colleague at Ruth's Chris to listen to the new, updated Oracle Warehouse Builder presentation.

For some background information, I have a book about OWB on my bookshelf and the first version of the tool never really took off, and I very rarely get inquiries about it, but they do come from time to time so I thought it would be a great opportunity to give the tool another look.

The meeting was from 11 AM to 3 PM and I arrived at 10:30 AM. There were about 20 cars in the parking lot. For reference, last month Information Builders held a lunch at the same location going over their new BI product, there were about 50 cars and a good 60-70 people in attendance, and it was a great lunch and good information. Back to the story.... I walked in the door and went to the registration table. There were a couple suits walking around eying everyone but not saying much. The lady at the table didn't have my information, but took my card and gave me a nametag and directed me to the breakfast area. I noticed about 20 people sitting at tables.

The whole place had the feel of a funeral. It was quiet and no one was saying much except two guys talking about flying a lot between Ft. Lauderdale and Tampa. My goal was to hear about the product and make at least 3 contacts, so I decided to join their conversation. A great segway was about flying. One guy mentioned that he was flying on that route 2-3 times a week on Southwest; I often did the Tampa-Miami run on American, sometimes 2-3 times per week. My record was 5x in one week last year. So this is a good point in common so I walked up, recounted my story, they laughed and sympathized with me and introduced themselves. The guy said he worked for Oracle and asked me what I did, I told him BI consulting. What he said next astounded me.

"You have to leave". I laughed and resumed the conversation but he just looked at me. I said "Is this some kind of joke?" He said "No, serious I need you to leave you can't attend the meeting". No one else was really saying anything and the Oracle guy wanted me to walk to the door with him so we could talk. After we walked outside, I asked him why Oracle would have listed on their website a BI briefing and then ask people to leave. He replied that this event was paid for by a consulting firm and they instructed that anyone not a client or potential client be asked to leave. He said that it's a sign of the times and the client paying is afraid of other companies stealing clients. Fair enough, I left just kind of surprised by the whole deal.

Interestingly, my colleague (a friend in the business, not a co-worker) was sitting at the table talking to people enjoying his breakfast when he was asked to leave.

This was definately an interesting impression on the Oracle BI product, and not the one I expected to receive.

The good news was that I was able to attend Tom Kyte's presentation at HCC and ran into some former co-workers and it was good chatting and listening to them.

Sunday, March 29, 2009

Orlando Code Camp Review

Yesterday I gave a presentation at the above mentioned event and it was the best one I have done so far. Had a great room, good audience participation, I felt awake and alive, and everything seemed to flow quite smoothly. I want to thank the Orlando organizers for hosting another great event. I look forward to my visits there, between the fall SQL Saturday and spring code camp, it's one of the highlights of my talks each year.

Thursday, March 19, 2009

Sprint Phone Comparison

I thought that today I could write a bit about cell phones. I prefer to use PDA phones since it's easiest to keep up with the business from these phones with respect to emails and such, and to disappoint a few, no I do not use an iPhone nor have any plans to get one.

I was using a Sprint 6700 until August of last year, when it failed me on a business trip. Dealing with refurbished models isn't something I enjoy, so I bought a Sprint Mogul brand new off Ebay. The Mogul was a good phone, I really liked it but it had issues with memory management. The phone would also shut off at random times, leading to missed calls and interrupted calls. I downloaded all the software updates but still ended up doing a hard reset on a daily basis. As a last ditch effort, I found a custom WM6.5 rom on the internet and loaded it on the phone. I must say that WM6.5 is a great software update and I can't wait for the official release, but it was too much for the Mogul to handle, and required twice daily hard resets to clear the memory.

Sprint offered to upgrade me to a Touch Pro, the newest line in PDA phones they offer. Because the price was right, I elected to try this. It meant that I had to turn in my Mogul though, as it was an exchange. My first thoughts of the Touch Pro is that the screen is beautiful, lots of resolution, but i miss the wide stance of the 6700 and Mogul. The Touch Pro is a little thinner, more the size of a traditional cell phone, but I have become used to the wider phones and that has taken some getting used to. The functionality is flawless, the phone works, email works, and it has plenty of memory which is much appreciated. The form of the phone seems fragile though, I can't help but think that one drop and it's toast.

For now, I'm pleased to have a phone that works, but I find myself missing the Mogul. I just found a Wm6.5 rom for it so I'll do my own software update and try to give this one a month to see if my satisfaction improves. If not, it will be back on Ebay and I'll move to something else.

Tuesday, March 17, 2009

The Future of ETL

Yesterday I wrote about the arduous task of installing IBM Websphere DataStage. I mentioned that DataStage has been the tool of choice for high-performance ETL, and I wanted to elaborate on this topic today.

In my talks that I do at Code camps, I always take a minute or two to talk about ETL tools. This is a very important part of data warehousing, perhaps the most important part (aside from the data). Too often, the ETL tool choice is made after hearing sales pitches but before really studying the differences in the tools.

There are three main tools on the market: Informatica, DataStage, and SSIS. I haven't directly worked from Informatica, but I'm told that it's similar to DataStage without the hashed files (Server edition). DataStage has some very, very high performance characteristics when using Enterprise edition with large data volumes on a partitioned tables/database design.

However, Microsoft SSIS is really coming up on the inside performance-wise. The 2005 version was very good and offered bulk loaders to load data, and the 2007 release of the native connection packages for different databases is a huge step in this direction. The 2008 version of SSIS is even better and has the cached lookup, which is a big leap for performance as well.

I've written before (in 2006) about some misses in SSIS such as the lack of an easy way to create surrogate keys. I know it can be done using scripting, but there isn't a transform function to do this (or use sequences/identity columns). However, these small misses shouldn't be enough to not consider the tool.

I believe that SSIS is the future of ETL as far as a platform because the price point and the features combine to create a powerful platform. As SSIS grows, other tools will become obsolete and SSIS will take a larger and larger portion of the ETL product market.

Monday, March 16, 2009

Installing IBM Websphere DataStage 8.0.1

Howdy folks, I thought today I would write about my installation of the IBM ETL toolsuite on my server.

IBM Websphere DataStage v8 is the newest version of the IBM ETL toolsuite. DataStage has for years been the leading high performance ETL tool in the market. DataStage was created by Ardent Software in the mid-1990s, Ardent was renamed to Ascential at some point, and Ascential Software was purchased by IBM for a tidy sum in 2004. In 2005, DataStage 7.5 was released as an Ascential product, and version 8 (v8) is the first true IBM release of the suite.

IBM has made a bunch of changes to v8 to integrate it more with their WebSphere web services product and to position DataStage as a software-as-a-service model more than the traditional batch product. For those of us just looking at DataStage as a nightly run process, this is a negative thing. For those trying to integrate DataStage into webpages, real-time data integration, and right-time data warehouse loading, it's a positive thing.

I installed DSv8 for a client on thier machine last year. I've installed DSv7 on Unix machines and Windows as well. I was able to trade a couple hours of database work for an extra server that one of my clients had sitting around, and I feel it was a great trade. The server is a HP pentium itanium 2 gz x 2 processors with 2 GB of ram. It's slow by today's standards, but it is a 2004 model. The worst part is that it doesn't have a DVD drive or USB2.0 which made putting DS on it a lot of fun.

First, I had to start with a fresh O/S wipe. I installed Windows Server 2003 Standard R2. This took about 45 minutes. I then took a couple hours to transfer the DataStage files over from my laptop using an external HD.

For those curious, I purchased a software package from IBM that includes a development license of DataStage v8. It wasn't cheap, and it's not easy to find, but it's there. I wish that IBM would make it easier and the license terms less onerous. I don't have an extra $350k sitting in my software budget.

I used my laptop to read the installation manual instead of printing it out (saving the environment), and it was pretty intense. There are a lot of settings, permissions, users, and environment variables to configure. I then ran the installation package for DS, which installs DB2 9.1 for use as the metadata repository. Once DB2 is installed, the actual DS installation commenced. It ran flawlessly to completion, and took about 75 minutes in all.

Once I had the software set up, I had to start the websphere server. However I still wasn't able to log into DS. Puzzled, I began to research. I discovered I needed to configure the user accounts in the Administrative console, and while it wasn't easy, it wasn't that hard either. There are a lot of screens and options in this area.

I was then able to log into the Designer and create my first ETL package on my home server. I'm still trying to detemine how I will handle my servers, so I'll write more about server management at some other time.

For now, happy coding.

Wednesday, March 04, 2009

New Website Functional

Hello all, I have a good announcement today

The relaunced website is up. The address is www.thedamndata.com. For now, this blog will still be hosted at http://thedamndata.blogspot.com.

The main addition to the website are two major components: Forums and News

The Forums are a discussion area that can be used to discuss anything from SQL databases to your best friends' dog fetish. Just kidding on the second part, but there should be a post for every want and need relating to data. If you feel we're missing something, please let me know and I'll make adjustments to them.

I'm searching for two moderators for the forums, please email me if you're interested.

The other exicitng part is News. The news feed is still being finished but news regarding data will be posted and continuously updated.

I hope you like and use the new website. Please refer your friends as well.

Thanks for the patience as the website was redone. I'm not a graphic designer and I'm sure there are a lot of changes in store for it in the future.

Friday, February 20, 2009

Market Update

Hello again, I wanted to spend some time today giving a pep talk on the economy. It's a tough time, right? Lots of local events (SQL Saturday, South Florida Code Camp) that I've attended lately have taken on a ghastly pall as tech people talk to those who are out of work or those who are afraid they might be. I can say from my perspective that people aren't in as good of a mood as they were a few years ago. The lack of recruiters at the events is a telling sign as well.

My training as a an economist will help me put another twist on this topic. My small business in software has been hurt a bit by the economy a bit. I know some poeple who have been laid off. I talk to customers or potential customers daily. Things have definately slowed down in the market. But where does this go?

UP! Up, you ask? How? Have I lost my mind? Some would like to say yes, but I really don't think so. In my discussions with clients noted above, I have yet to talk to a single client who has cancelled a project. Most of the managers are cautious and content to push them into later in this year. Budgets have been crimped and a lot of money isn't available as it used to be. YET the work and the necessity of the work is not diminished. This bodes very well for the future of IT.

It's really a shame that many companies and managers view IT as an expense and not an asset. My company specializies in Business intelligence, this is a raw asset. Every penny that you invest in my company building your BI tools will easily return it's cost, and if it doesn't do significantly more than that I have failed. Some parts of IT have expense components - but the net results are assets.

For a look at the numbers, the official unemployment rate is about 8%. That means 92% of people are working. Economic growth is always sluggish in the winter months. It might be a slow spring and hopefully a better summer, but the work is out there, it's building, and the best thing one can do is to use this recession as a time to build up for the coming good times. Build your network, work on self-training. Read that SQL book that you're had for three years on your bookshelf. Attend a lot of the free events in our area, meeting new people, learning about new technologies that are going to be key in the next boom cycle.

A recession is a horrible thing to waste.

Friday, January 02, 2009

Website Update

It's a new year and time for an update to this website.    

While the update is in progress, I'm going to turn off the blog, so I've put my next posting live a couple days early.  I hope to have the upgrade done next week so be sure and check back.  It will probably take a couple of weeks to get the kinks worked out.

Thanks for the support.  

A Positive Story from the Field

Yesterday I relayed a story about a consultant who made a process improvement of little value to the client and in doing so, created a lot of headache.  Today I'll recount a positive story of a value-added process improvement that made a noticeable difference.

One of the clients I was working with asked me to take a look at their database performance.  They felt that it was pretty poor and they were right.  One of the things I noticed immediately was the over-analysis of tables.  To explain, they were using Oracle and had an ETL job that analyzed the tables as soon as the load job finished.  The particular instance had a small amount of daily updates but the statistics were going stale on a daily basis.  When I started looking at the load job, I found that it did not differentiate between true and false updates.  A true update is when a data component changes, and a false update goes down the load option without being changed from its' prior state.  

Generally in good ETL architecture, the architect will put an escape clause to only load the true updates by doing a before and after comparison.  This keeps the load process free from only changing the update timestamp column as was happening in this instance.   

We had to change this job to only load the true updates in the target table, and in doing so we solved the stale statistics problem and also cut the load process by a factor of 10x.  By changing this job we were able to disable the analyze table routine thus saving more processing time.   The client was quite happy in this instance as the total time savings will be in the hours by the time this architecture change is propagated across all their subject areas.

Seconds don't sound like much until they begin to aggregate over a period of hours.  In my next post, I'll talk about separating DBA maintenance tasks from ETL processing.

Until then, happy data warehousing.  

Stories from the Field, Part 1

As I promised back in September, I would write about some experiences I've had with consulting.  I expect there to be many parts to this title, so I'm going to set some ground rules.
1. There will be no client names mentioned (or person names).
2. No part of my posts should be interpreted to insult a client or person, I am only reciting the event as it happened.
3. Names may be modified to protect the parties involved.

Now that the disclaimer is done (and I mean it, especially #2), let's start with our first story.

I was on a team working with a few other consultants on a large data warehouse project.  One of the guys (not brought in with us) was an independent consultant.   Let's call him Rob, and Rob is a good guy, very interesting and funny, and his stories caused great laughter at lunch.   Rob had a way of being very straightforward and profane, a quality that does not go well when working at client sites.  It was difficult for him to say a complete sentence that didn't include the F-bomb or using the G-D word, or any other curse word you can imagine.  This 'quality' frustrated a lot of the employees and gave me great pause into engaging him in conversation in front of other people. 

One day in a meeting Rob had a great idea.  In the client's ETL jobs, all jobs were designed to terminate at the first sign of an error, send a page to the duty pager, and have manual intervention to restart the process.  Without getting into whether this is good architecture or not, it is what it is, and Rob had an idea that we could create a routing using an exception handler in all ETL jobs instead of just terminating them.  He said it would be easier to code the jobs this way, and in a way he was correct.  Rob's problem was that he was solving an issue that wasn't a problem for the client - the client was happy with the way the jobs previously functioned, and Rob's hours and effort to retrofit working jobs with this new technique took valuable time away from solving problems that the client had identified.   However, the manager told Rob if he could get a sample done quickly, he would be open to changing the architecture.  

A couple of weeks later and Rob presented his working idea at a team meeting.  I had cautioned him in private about solving a problem that the client didn't seem to have, and my colleague suggested to him that this approach didn't take into consideration all the possible failure scenarios.  Rob pushed on and in the meeting, my colleague again suggested in front of everyone that he wasn't convinced this technique would work as described, and cautioned Rob to test it with every possible scenario before putting this into production.    The manager approved this code to be moved to production only after he had some tests for every scenario that had been described by my colleague.

By now you are probably starting to imagine the outcome of this scenario, and if your outcome includes the word disaster you are correct.  Rob moved this code into production and the first night there was a hiccup, but because he hadn't tested correctly for that failure scenario, incorrect data was loaded into the production tables and the job did not abort as it should have.  The next day people around the company were noticing data that was incorrect on reports and poor Rob had some explaining to do, which he did in a status overview laced with enough profanity to make a sailor blush.  

In the end, after much testing and some more issues, Rob was able to make this routine work correctly but the client largely considered it a failure and chose not to move forward with this approach, now leaving one subject area with this architecture while the remainder used the legacy code.  

For this story, let's analyze what happened here.  Using the flowchart methodology,
Rob identified a process improvement that the customer did not value and Rob did not adequately test this process improvement, causing the customer to suffer delays and bad publicity in their company, thus causing them to be shy on implementing further process improvements.

I'll make the next story an example of success, but the moral of the story here is that if you are going to go out on a limb to perform a process improvement, make sure it's something that the customer values and last but not least, make sure that it works.




Thursday, January 01, 2009

A Year in Review

First I would like to wish everyone a happy new year.  In many ways it's difficult to comprehend that another year has come and gone, a year that was challenging for many.   I am pleased to say and say it with some trepidation that 2008 was an excellent year for myself and Durable Impact.  DI is making some real progress in acquiring projects and clients and now I have a very good springboard for launching into 2009.  

One thing I didn't quite accomplish this year is giving a lot of presentations.  By my count, I only did five events this year.  I was very busy building DI up and traveled about 32 of the 52 weeks.  Traveling is not conductive to building a brand by giving speeches, but it is better to travel and make money than to give talks for free.  I have been and will always try to find a happy medium.  

SQL Saturday last February and the Day of Data event were great successes.  Pam Shaw and I are hosting another SQL Saturday on January 24 at the KFORCE facility in downtown Tampa.  Unfortunately, I was not able to put together a Day of Data to go with the Saturday, so maybe I'll host one in the spring. 

With 2008 we saw a launch of SQL Server 2008 and some great improvements to the SSIS tool suite.   With the coming launch of Windows 7 in 2009/early 2010, it's looking to be another good year for Microsoft. 

As for my personal goals, I'm just interested in working, building up DI, helping out the community where I can, and having a general all around good time.  If you're ever in Tampa and want to chat, drop me a line through the DI website (linked on the left) at the Contact Us form.   I will also try to put up more blog postings, as I have a fairly strong amount of readership and don't like to disappoint.  
 
Here's to a great 2008 and an even better 2009.