Leaky Abstraction & Dataset Serialization

So here's my rant on datasets and leaky abstractions. Datasets are powerful; they come in with a lot of pre-built features such as versioning, and relationship building. However, one of the fallacies of datasets is that they serialize via XML when sent across the wire via Remoting or web services. Why is that bad? Well it’s really bad when you have large amounts of data in the dataset. Serializing into xml means converting the data that was in the dataset to strings and wrap descriptive metatags around the data. Depending on the column name the metatags can easily add 50% overhead to the size of the dataset. Not to mention that a 10 digit 4-byte integer when converted to a string now is at least 10-bytes (depending on encoding) But you don’t have to worry about that since .NET has a Garbage Collecter and should clean it up when you’re done right? Well, that depends, if your object reaches the threshold of 85kb it gets put on the large object heap, which rarely collects, and large object heaps work in similar fashion to the traditional malloc() way of allocating memory. Which means it’s possible to get OutOfMemory exceptions due to memory fragmentation when you actually have the free memory available. I believe this is one of Joel’s leaky abstraction scenarios. So what should you do if you HAVE to send that much data across? Well usually I would say just break the data up and have a bit more chatty less chunky communication, however if you absolutely have to, you can use dataset surrogates instead. Basically the surrogate copies the data and relationships of a dataset into the surrogate object and serializes in binary.

Here is the KB Article on Dataset Surrogates:

http://support.microsoft.com/default.aspx?scid=kb;en-us;829740

Also along the same line, this guy went all out on to demystify and explain the performance benefits of Remoting vs Web Services.(You'll notice there is a big difference between dataset vs custom objects when sending it across the wire, wonder why...)


http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/bdadotnetarch14.asp

And here’s an excellent 2 part article on how the .NET garbage collection works. To sum it up, basically the GC compacts the heap to prevent memory fragmentation (gasp!) this also allows for fast initialization. When GC performs collection, all threads are suspended, and it searches the heap for objects without a reference, any remaining objects are elevated to another generation. The higher the generation an object the longer it will stay in memory without being cleaned up, that is why calling GC.Collect() all the time is a BAD idea. You may end up elevating objects with a short lifespan and they will not get cleaned up in a timely manner and eventually eat up your memory.


http://msdn.microsoft.com/msdnmag/issues/1100/GCI/default.aspx

Custom Configuration

Haven't posted in a bit, but added a new project today to the website. Basically it's on how to write your own custom configuration handler, incase you want some simple light customization to your config file. If you want full blown OOP config I suggest you go look at Enterprise Services, Configuration Application Block, but that's a little too heavy for me. =)

C# .NET Text Insertion

So this is an old project, basically with .NET it makes things easier to code, due to all the helpful libraries and helpful utilities. However there are still some things that you should NOT do, such as loading a 200MB file into a string so you can call string.replace or myString.Insert(15,"new string") so as to save it back into a file.(It amazes me that things like this still happen) So how do you insert a text string into the middle of a file? You would have to do it the old fashioned way and move blocks of bytes around so as to insert your string. (I'm not sure why StreamReader/Writer doesn't have this feature, either way a programmer should be able to implement this themself)

SQL Existence Check Statements

So as I was working with some SQL today, I realize that when it comes to writing deploy scripts and DDL scripts, it is useful to use these SQL check clauses to check for the existance of the object before dropping and recreating. Although these scripts might be obsolete with the advent of meta tables in later version of SQL, it's still useful as a reference.

--Checking Table Column
if exists (SELECT * FROM sysobjects o, syscolumns c WHERE o.name like 'TableName' AND c.name like 'ColumnName' AND o.id = c.id AND (o.type not in ('P', 'FN', 'TF', 'IF') OR (o.type in ('TF', 'IF') and c.number = 0)))

--Checking Stored Procedure
if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[ProcedureName]') and OBJECTPROPERTY(id, N'IsProcedure') = 1)

--Checking Functions
if exists (select * from dbo.sysobjects WHERE id = object_id(N'[dbo].[FunctionName]') and xtype in (N'FN', N'IF', N'TF'))

--Checking Tables
if exists (select * from [dbo].sysobjects where id = object_id(N'[dbo].[TableName]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)

--Checking Views
if exists (select * from [dbo].sysobjects where id = object_id(N'[dbo].[ViewName]') and OBJECTPROPERTY(id, N'IsView') = 1)

ASP.NET 2.0 Master Pages Modifying Header

So with the introduction of ASP.NET 2.0, one of the newest and most hyped about feature is Master Pages. With master pages, one can easily create web templates for their website. The basic idea is you create a template known as a master page, and you set certain spots in your template html code as place holders. Then when you create an aspx page you define which is your master page. Inside your aspx page, the placeholder controls will be the only controls in your aspx page, and whatever code goes inside these placeholder controls will show up where they were placed in the master pages. All that is fine and dandy except, you can't use this for the header part of the html, meaning the placeholders can only be placed between the <body></body> tags of the master pages but not the <head></head> . So what happens if you want to add some meta tags, or some link and script tags?? Well so far the only way I discovered this to be possible is to create html controls in your code behind and add them to your Header object. I have included a code snippet below:

HtmlMeta oMeta;
oMeta = new HtmlMeta();
oMeta.Attributes["http-equiv"] = "Content-Type";
oMeta.Attributes["content"] = "text/html; charset=UTF-8";
this.Header.Controls.Add(oMeta);
HtmlGenericControl oScript = new HtmlGenericControl("script");
oScript.Attributes["language"] = "javascript";
oScript.Attributes["type"] = "text/javascript";
oScript.Attributes["src"] = "../Scripts/script.js";
this.Header.Controls.Add(oScript);

So what this code does is that it create a new html meta object, sets the http-equiv attribute to content-type and sets the content to text/html; charset=UTF-8. This informs the browser to read the text as UTF-8 encoded. (this is useful if your website is in a language other than english, and you tell the browser what kind of encoding it is) The code also creates a script tag and adds it to the header, I have not found a HtmlScript control so I used an HtmlGenericControl to build the script tag. This is helpful if you want different pages to have different scripts involved. So this is how you add items between the <head></head> tag in your pages that work off of a master page.

CodeDom and GDI+

So today's post is about a small learning tool that I worked on. Basically it concerns two aspects of .NET that I find quite interesting. The first part is CodeDom, which lets you generate and compile code during run-time. Such as off of a file, string, or Document Object Model (DOM). So that you can programatically create class, variables, and methods and then compile it all during run-time! Very cool if you ask me, it gives you the ability to easily write an interpreter, or scripting engine. The other half/purpose of the program is so I can learn more about GDI+ and the available methods it offers. Basically what my program does is that, you can write code into a text box (preferably GDI+ code) and it'll run the code and render the graphics onto a picturebox. The original intent of this tool really was for me to draw rounded corners for my website. Since I'm a programmer I figured writing code might be easier for me than dragging a mouse. But in the end, what had happened was that I used this and found how GDI+ rendered the graphics on the screen and mimicked the results in photoshop. So essentially I used a combination of this and photoshop to help me create images. Nonetheless it was a good learning experience. I posted a page on my site about this project and some explanations. If I find more use for this program then I'll improve on it, but in the mean-time it serves as a good code sample/repository for me.

C Coding Problem

So I couldn't help but stop thinking about this simple coding problem I was presented with. What I have down is what I think is the answer although I have not tried to compile the code. I don't believe I solved the problem in-time but at least now I have an answer written down for next time.
Input: He^ll^^^oW^or^ld^^^^^
Output: HelloWorld

So the goal is to write a function that processes the input and returns an output as mentioned above. I accomplish this by reading through the characters once, and using two pointers to keep track of what I'm reading and what I'm writing.
void func(char* p)
{
char* currread = p;
char* currwrite = 0;
while(currread++)
{
if(!currwrite &&amp;amp;amp;amp; *curread=='^')
currwrite=currread;
if(currwrite && *currread!='^')
{
*currwrite = * currread;
++currwrite;
}
}
currwrite=0;
}

This is probably the easiest to visualize and follow solution, another way is to use strcpy and copy the strings to currlocation from currlocation+1 to length-currlocation each time a ^ is encountered. From my discussion with a cohort, window's implementation of strcpy boils down to only a few assembly instructions so that may turn out more efficient.

A new beginning

So I decided to work on a personal site, and start a blog on it. This will help keep track of the things I have learned while making the site as well as any other technical things I have learned in general. In essence it is a weblog of my technical life. Occasionally I'll go back to things I've already discovered in the past just so it can be documented and so I can personally search it up.