Nerdliness.com

One of our most popular posts on this site, at least in terms of search engine traffic, is our Using ASP.NET AJAX CascadingDropDown With Visual Basic and a Database article from February 2008.  Long story short, in that article I broke down a Microsoft ASP.NET AJAX tutorial, translating their C# code into Visual Basic and used a MS SQL database for the datasource.  Good times.

Seems that some folks out there, people more inclined to use C#, actually like OUR example better and have spent time translating our code back into that language.  Oh, how the tides have turned.

One of those C#ers, a Mr. Bradley Hall, was kind enough to post his C# version of that sample code in the comments of that post.  With his permission, I'm throwing it up here, too.  Thanks again, Bradley.


using System;
using System.Web;
using System.Web.Services;
using System.Web.Services.Protocols;
using System.Collections;
using System.Collections.Generic;
using System.Collections.Specialized;
using AjaxControlToolkit;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;

[WebService(Namespace = "http://tempuri.org/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
// To allow this Web Service to be called from script, using ASP.NET AJAX, uncomment the following line.
[System.Web.Script.Services.ScriptService]
public class nbaplayers : System.Web.Services.WebService {

[WebMethod]
public CascadingDropDownNameValue[] GetConferences(String knownCategoryValues, String category)
{
String strConnection = ConfigurationManager.ConnectionStrings["nerdlinessConnection"].ConnectionString;
SqlConnection sqlConn = new SqlConnection(strConnection);
String strConfQuery = "SELECT * FROM CONFERENCE";
SqlCommand cmdFetchConf = new SqlCommand(strConfQuery, sqlConn);

SqlDataReader dtrConf;
List myConferences = new List();

sqlConn.Open();
dtrConf = cmdFetchConf.ExecuteReader();

while(dtrConf.Read())
{
String strConfName = dtrConf["conf_name"].ToString();
String strConfId = dtrConf["conf_id"].ToString();

myConferences.Add(new CascadingDropDownNameValue(strConfName, strConfId));
}

// Cleanup objects.
cmdFetchConf.Dispose();
dtrConf.Close();
dtrConf.Dispose();
sqlConn.Close();
sqlConn.Dispose();

return myConferences.ToArray();
}

[WebMethod]
public CascadingDropDownNameValue[] GetTeams(String knownCategoryValues, String category)
{
String strConnection = ConfigurationManager.ConnectionStrings["nerdlinessConnection"].ConnectionString;
SqlConnection sqlConn = new SqlConnection(strConnection);
String strTeamQuery = "SELECT * FROM TEAM WHERE conf_id = @confid";
SqlCommand cmdFetchTeam = new SqlCommand(strTeamQuery, sqlConn);

SqlDataReader dtrTeam;
StringDictionary kvTeam = CascadingDropDown.ParseKnownCategoryValuesString(knownCategoryValues);

Int32 intConfId;

if (!kvTeam.ContainsKey("Conference") || !Int32.TryParse(kvTeam["Conference"], out intConfId))
{
return null;
}

cmdFetchTeam.Parameters.AddWithValue("@confid", intConfId);

List myTeams = new List();

sqlConn.Open();
dtrTeam = cmdFetchTeam.ExecuteReader();

while (dtrTeam.Read())
{
String strTeamName = dtrTeam["team_name"].ToString();
String strTeamId = dtrTeam["team_id"].ToString();

myTeams.Add(new CascadingDropDownNameValue(strTeamName, strTeamId));
}

// Cleanup objects.
cmdFetchTeam.Dispose();
dtrTeam.Close();
dtrTeam.Dispose();
sqlConn.Close();
sqlConn.Dispose();

return myTeams.ToArray();
}

[WebMethod]
public CascadingDropDownNameValue[] GetPlayers(String knownCategoryValues, String category)
{
String strConnection = ConfigurationManager.ConnectionStrings["nerdlinessConnection"].ConnectionString;
SqlConnection sqlConn = new SqlConnection(strConnection);
String strPlayerQuery = "SELECT * FROM PLAYER WHERE team_id = @teamid";
SqlCommand cmdFetchPlayer = new SqlCommand(strPlayerQuery, sqlConn);

SqlDataReader dtrPlayer;
StringDictionary kvPlayer = CascadingDropDown.ParseKnownCategoryValuesString(knownCategoryValues);

Int32 intTeamId;

if ((!kvPlayer.ContainsKey("Team")) || (!Int32.TryParse(kvPlayer["Team"], out intTeamId)))
{
return null;
}

cmdFetchPlayer.Parameters.AddWithValue("@teamid", intTeamId);

List myPlayers = new List();

sqlConn.Open();
dtrPlayer = cmdFetchPlayer.ExecuteReader();

while (dtrPlayer.Read())
{
String strPlayerName = dtrPlayer["player_name"].ToString();
String strPlayerId = dtrPlayer["player_id"].ToString();

myPlayers.Add(new CascadingDropDownNameValue(strPlayerName, strPlayerId));
}

// Cleanup objects.
cmdFetchPlayer.Dispose();
dtrPlayer.Close();
dtrPlayer.Dispose();
sqlConn.Close();
sqlConn.Dispose();

return myPlayers.ToArray();
}

}

Now I hate to insult Bradley in any way, but I should point out that this is his code, not ours, and that if you aren't a C# aficionado you'll have to trust him that it's does what you'd expect. And, of course, use it at your own risk.

Disclaimer aside, it looks great by my limited C# experience, and again...  thanks, Bradley.

Traffic here at Nerdliness is generally pretty level.  Sure, there's the occasional spike on days we post new content, and the overall trend is in the upward direction, but viewed over a monthly timeline the graphs are roughly flat.

A couple of weeks ago, I was doing my daily OCD-ish perusal of the Nerdliness.com Google Analytics reports and started seeing something odd:  a substantial bump in visitors.  My first thought was, of course, that Someone Important had recently discovered our sheer awesomeness and was preaching our gospel, but reality soon set in.

After investigating a little further, I noticed that the Content reports showed that all this traffic was directed at a page that doesn't exist.  I looked through our Apache access and error logs, trying to find any references to those URLs and came up with nothing.  

On the off chance that I might have actually made some sort of mistake, I also compared the Google Analytics report to our Google AdSense data.  Our 404 page has a couple of AdSense blocks on it, so I figured that, if it was the result of a bad link and visitors getting a 404 error, we'd see a proportional increase in AdSense impressions.

Nothing.

Looked around at every report, log, and portent I could think of and found nothing.  Double-checked our AdSense code snippet, looked good.  At that point, I was reasonably sure that it wasn't on our end and started to suspect that, perhaps, someone else was using our GA code.

So I emailed Google Analytics support. 

(Quick side note:  GA support contact link was a pain in the pooper to find.  If you ever need to contact them yourself, you first have to go through their Google Analytics Troubleshooter and just through a few hoops.  Assuming you don't find your answer, you should get a contact form at the end. 

Good news is that they were very quick to respond, replying to my ticket well within the promised 24 hours.  Good on you, Google.)

GA support didn't 100% confirm that someone was using our code, but their canned message did suggest that was the most likely cause:

Finding information about traffic to domains that are not yours in your reports is possibly the result of someone accidentally entering the wrong code on their own site, or borrowing/displaying some of your website's code for their website.

Even better, they included a possible fix:

If you're concerned about this data corrupting your own reports, Analytics can easily filter on a specific domain so you can avoid this problem. We recommend creating an 'Include' filter on your own domain:

Filter Type: Custom filter > Include
Filter Field: Hostname
Filter Pattern: your-domain-name.com
Case Sensitive: No

Sweet.

In case you don't know, you get to the Filter setup by:

1. Log in to Google Analytics (duh).
2. Click the Edit link for the domain you want to Filter.  It's in the far right column, under the Actions heading.
3. Click the "+Add Filter" link.  It's in the third section down with the heading "Filters Applied to Profile."  The link is on the far right, on the same grey background as the heading title.

Once there, just follow the Google Analytics Support instructions above.

Now, it doesn't look like the filter will do anything to historical data, just info that comes in after the filter is in place.  That said, I think that our new Google Analytics setup will include setting up this input filters as soon as we set up a new domain, just in case.

I love sharing my screw-ups with you all.  One of my favorite things, right up there with raindrops on roses and whiskers on kittens.  It's therapeutic for me and, hopefully, helpful for you in the God-I-hope-I-can-avoid-HIS-idiotic-mistakes way.  Win/win.

Thankfully, it happens so often I've got enough material to keep this blog afloat for months.  Take our recent Oracle issues at the office.

We have a fairly large Oracle database in one of our remote offices that's home to about 200GB of GIS data.  In addition to that production version, we maintain a development copy of the data for, well, development. 

Occasionally, we need to move data between the two instances, and my prefered method for doing so involves using the Oracle Datapump utilities (expdp for exporting, impdp for importing).  My process looks something like this:

1. Dump source database to disk via expdp.
2. Copy dump file to destination server.
3. Drop the schemas I need to refresh.
4. Import the specific schemas via impdp.

I could just export the specific schemas that I'm interested in, but I like to get the full backup just in case there's some object I forgot I needed, etc.

Couple quick explanations for any non-Oracle types out there.  First, in Oracle, a "schema" and a "user" are essentially interchangeable terms.  Schemas own objects, can have roles/privileges granted to them, etc. 

For example, the TestUser schema could be granted the CONNECT role, thereby giving it the ability to, well, connect to the database.  The TestUser role might also be granted the ability to create objects like tables, views, etc., so you might find yourself querying the TestUser.TestTable table.

Second, Oracle has an object called a Sequence that doesn't exist in certain other databases (<cough>MS SQL<cough>).  Like the name implies, a Sequence is an object you can use to create unique, sequential-ish* values to insert into tables.  Something you might insert, for example, into an ID field.

Got it?  Everything making sense?  Ok, on to My Screw Up.

The other day, I needed to refresh several Schemas in our Dev environment with Production data and, as usual, I did so using those 4 basic steps I mentioned earlier.  Export, copy, drop, import.  Seemed to work like a champ.

Until, that is, users started playing with Dev again.

Suddenly, they'd receive various contraint violations when trying to insert into tables. In a nutshell, some of the tables had unique constraints on ID fields so a user or application couldn't accidentally insert records with the same ID.  The users and applications would get those ID values by using the NextVal function of existing sequences.

Now, as I mentioned, Oracle GUARANTEES that a sequence will never ever give out the same value twice (unless, of course, you went and fiddled with the Sequence by reseeding it, dropping/recreating, etc).  And yet, a database that worked fine in Production seemed to be doing just that. 

Or rather, it seems that somehow the Sequences and Tables using them were getting out of sync.

The weird thing was that it didn't happen all the time.  We had some 2000 sequences in this database, and not all of them were acting like this.  Some applications would work fine after one database refresh, and fail completely after another.  No rhyme or reason.

We hit our heads against this for days, trying to get to the bottom of it before we figured out what was happening.

Remember how I mentioned that the Production database was pretty large, about 200GB in size?  Well, with our hardware, Oracle Data Dump would take about 1 hour to completely export the data.  And, because we were lazy, we were performing the export during normal working hours, meaning that users were actively hitting the database during that hour long export process.

Furthermore, after looking at the export logs, we realized that the expdp process dumped out the Sequence data toward the beginning of the process and dumped the table data toward the end.

See what was happening?

Let's say that we started the export at noon.  In our mythical database, we've got a TestUser schema with a TestSequence object used to populate the TestId field in TestTable. 

Before the export begins, TestSequence's Current Value would be 100, it's NextVal would be 101, and the MAX(TestId) in TestUser.TestTable should be 100. 

So the export kicks off and dumps the TestSequence info to disk at around 12:01.  It then goes on to dump other objects to disk.  Constraints, views, etc. 

Now it gets to the tables, maybe 15 minutes later.  It starts processing those tables, starting with the largest ones.  TestUser.TestTable is a smaller table, so maybe expdp doesn't dump it to disk until 12:45.

Meanwhile, endusers have been actively using the database and inserting rows into TestUser.TestTable, using TestUser.TestSequence to get the TestID data.  So maybe now TestSequence's Current Value is up to 105.  More importantly, the MAX(TestId) in TestTable is also up to 105 when it's finally written to the dumpfile.

Now, when I copy that dump file over to the destination server and import it, it imports TestSequence with the data it had when it was first exported (CurrVal: 100, NextVal: 101) and the TestTable with the data it had when it was exported (Max(TestID): 105).

See the problem?  When someone in Dev next tries to insert into the copy of TestTable, it gets the value 101 from TestSequence.  However, that table already had that value inserted.  User ends up with an ORA-00001: unique constraint violated message, and I get my lunch interrupted with phone calls.

Simple fix, of course, once we figured out the problem: don't export while users are connected.  Either schedule the dump to occur during after hour maintenance windows, or lock everyone out for that hour.

Alternatively, we could have taken a second export of the source database, one that only dumps the Sequence info.  Then, after importing the first, full file, import the second sequence-only one.

*Oracle guarantees that Sequence values will be unique, but not necessarily sequential. Has to do with how Oracle handles caching sequence values for different sessions, etc.  If you need the values to be sequential, you'll need to write your own function.

Last week, I was lucky enough to get an invite to test out a new "Social Search" application called Aardvark that was recently released into the wild.  Well, not completely into the wild (more on that in a sec), but wild enough that a no-talent hack like me managed to get an invitation (thanks again, @marshallk and @davidadewumi).

So here's how it works:

1. You sign up.
2. You tell Aardvark what types of questions you'd like to answer, rough categories like "Technology" or "Seattle."
3. People ask questions through their IM client (Google Talk, AIM, Windows Live Messenger).
4. Aardvark forwards that question along to a user on the network who's registered to answer questions about that topic.

You can tell Aardvark what times of day you're available to answer questions, and you always have the choice to Pass on one you can't answer.

A typical exchange might look something like this:

Asking questions is just as easy as answering them.  You just IM a question to the aardvark.im contact and poof, Aardvark does the rest. 

All in all, it's pretty damn cool, and extremely useful.  I can't wait to see what happens once they get more users with a wider knowledge base onboard.  Other than that, my only Wish List item would be Twitter support.

Now, getting back to that releasing-into-the-wild thing.  Right now, Aardvark isn't open to the public, so there are only a couple ways to get in on the fun:

1. See them at SxSW.  Unfortunately, I believe today is the last day for the Interactive part of that show, so it's probably too late for that option.
2. Find someone with invitations to share.  Someone, perhaps, who's devastatingly handsome... with a really nerdy blog...  who wrote this post...

So, yeah.  I have a few invitations still that I'd be happy to share.  Just email email me (justin@nerdliness.com) or leave a comment here if you're interested and I'll get back to you with the details.  I only ask that you a) actually use the service and b) name your next child after me.

Look, I know this isn't our normal type of post.  No code snippets, no real world examples, etc.  Barely even a trace of wit and/or style.

Ok, maybe the lack of style thing really is in keeping with our normal posts.  Whatever.

Point is that this is Important Stuff and, therefore, you need to know:

The deadline for submitting proposals for Open Source Bridge is fast approaching, as in you-need-to-get-yours-in-in-under-20-days-as-of-this-writing fast.  Or, more specifically, by March 31.

So what, exactly, is Open Source Bridge?  Well, my under-rock dwelling friend, it is:

Open Source Bridge is a new conference for developers working with open source technologies. It will take place June 17-19 in Portland, OR, with five tracks connecting people across projects, languages, and backgrounds to explore how we do our work, and why we participate in open source. The conference structure is designed to provide developers with an opportunity to learn from people they might not connect with at other events.

Them's their words, of course.  Me?  I like to think of it as The OSCON Replacement That Will No Doubt Be Better Than Its Predecessor, So San Jose Can Just Keep That Bloody Thing Because We Don't Need Them In Portland, Anyway, Conference. 

Granted, "Open Source Bridge" rolls off the tongue a little easier.

So go to the Open Source Bridge site.  Submit a proposal!  Volunteer!  Sponsor!  

And if you can't do any of those things, just make sure to register and get your bums to Portland in JUNE.