Wednesday, December 06, 2006

Reimaging a RAID array from ATA using Windows XP

Recently I found myself in the awkward position of needing to replace my laptop's ATA Hard Drive with a RAID array (RAID 0 - Striping). Generally this should be a happy time due to the HDD performance boost you could expect from a striped array. Unfortunately - I could not afford to loose too much time installing the drive and setting up the necessary software environment.



The obvious solution to this is to image the new RAID array using the old drive's contents and generally, this would be a relatively painless proceedure. RAID tends to complicate matters however, as windows requires RAID drivers at installation. Using an installation of XP from an image taken from a non RAID'ed drive results unpredicatable results (freezes, BSOD etc.) when imaged to a RAID'ed one. In addition, not all imaging software supports RAID imaging. Norton Ghost 2003 and earlier do not support RAID..... I used a trial version of Acronis True Image which worked like a charm.



To get Windows working, after applying the image to the newly installed RAID array, you can boot from the Windows CD and repair the imaged windows installation. Other drivers such as video drivers should probably be installed - I found my nVidia GeForce 6800 prevented me from booting into Windows. I could load Windows Safe mode with network support however.



Below is a list of the steps I took to install RAID and restore my previous software environment:




PREPARATION
  1. Installed Acronis True Image
  2. Ran a backup to an external USB HDD
  3. Created a bootable CD in Acronis
RAID INSTALLATION
  1. Copied RAID Drivers to floppy disk
  2. Booted into BIOS to check boot order (floppy, CD, hdd etc.)
  3. Cracked open the laptop and removed the old hdd.
  4. Set the jumpers on the new Slave hdd to cable select (as per specs for Alienware laptops)
  5. Installed 2 new hdds (of equal size, speed etc)
  6. Booted into BIOS and set the HDD Mode to RAID
  7. Save and Exit
  8. Boot into RAID BIOS (I used FastTrack.... CTRL-F at bootup)
  9. Include both the installed hdds to a Striped (RAID 0) array
  10. Save and Exit
  11. Verify on bootup display that the RAID array can be detected
REIMAGING
  1. Boot off the Acronis Bootable CD (PREPARATION - step 3)
  2. Follow the prompts to reinstate the image stored on the USB HDD - resizing the target partition size if necessary.
  3. Boot off the Windows CD leaving the RAID Drivers in the floppy drive (RAID INSTALLATION Step 1)
  4. Press F6 when prompted to install third party SCSI or RAID Driver
  5. Select the driver from the options given
  6. Proceed through Windows installation until asked if you wish to repair an existing installation of Windows
    -NOTE - Do not choose the Recover a Windows installation option at the start of the Windows installation process.
  7. Select Yes and select the installation of Windows you wish to repair
  8. Continue reinstalling Windows.
  9. Boot into Windows safe mode.
  10. Reinstall motherboard chipset, video and sound drivers.
  11. Boot into Windows and everyone's happy.

Thursday, May 18, 2006

Asp.net 2.0 Profiling - Auto Saving Form Data

-->

Microsoft has extended the standard web state persistence models with the addition of profiling in the ASP.NET 2.0 framework. Essentially using profiling is identical to using the HttpSession, the difference is that user data is persisted between site visits. By using the Profile, a website is able to save and retrieve persisted user data regardless of the user's browser / session / machine state. We are now able to persist user specific data (i.e. session information) between site visits without needing to write a data abstraction layer (db interface).

In other words, we can save session related information between sessions. Ignoring all the extra ASP.NET 2.0 Framework bells and whistles, an obvious use for Profiling would be found in UI design; saving and repopulating the entered data of Form input controls (i.e. Textbox) between a user's visits. One use for this sort of functionality would be remembering a User’s previous search in search pages. Another use would be to temporarily save a User’s progress through larger form based work processes (i.e. completing an online survey). This way, should the User’s Session be interrupted (i.e. browser crash), the User can continue data entry from where they last left off the next time they log onto your Site. User form data is automatically saved on postback without the need for special persistence code.


The example code (github : https://github.com/benkitzelman/ProfilePersistence) demonstrates how to construct a custom Profile responsible for transparently saving and restoring registered control values on Page loading.


By looking at PROFILE.CS we can see that it inherits System.Web.Profile.ProfileBase. It contains a public method ‘Register’ which takes a System.Web.UI.Control as one of its parameters. Register is responsible for persisting a passed control’s property to the profile, and registering it for repopulation on the next postback. The second parameter is a string defining a property name with which to bind to (i.e. when registering a TextBox, we could bind the “Text” property). Using the given control’s Page property, event handlers are assigned to the parent page’s Pre Render and Pre Loading events. Note that within the PreLoading event handler, profiled data is only loaded into the target control on the first load (Page.IsPostBack == false) preventing any overwriting of entered data between postbacks (ViewState).


The Profile manages a collection of UrlProfiles. Essentially a UrlProfile is a Serializable object (data container) responsible for managing profile data for a particular url. It is also responsible for managing the profiled data for registered controls within its Target Page.


The UrlProfile was implemented in preference of a Page Profile as quite often a page functionality can be quite varied depending on its Url (address + query string parameters). Each registered control has a ControlProfile constructed.


Ideally the reading / populating of target controls should be abstracted, requiring as little handling code as possible. Looking at PROFILETEST.ASPX, we can see that only one line of code at initialization is required to register a control for Profile persistence.



protected override void OnInit(EventArgs e)
{
base.OnInit(e);
Profile.Register(testBox, "Text");
}
The only thing remaining is to hook up the custom profile in the website's web.config specifying a default profile provider (see Custom Persistence Layer below):

Custom Persistence Layer
By using a provider model, how this data is persisted is highly configurable. The standard (implicit) provider is the SQLProfileProvider (all data is serialized and stored to db). One can however, inherit the System.Web.Profile.ProfileProvider base class and assume control of the data persistance In the example code, the TEXTFILEPROFILEPROVIDER.CS serializes User data to a text file. This implementation is based heavily on the MSDN Example.

Saturday, March 04, 2006

DTMF Sampling - Constructing a Wave

-->


How the heck do you generate dynamic sample data suitable for the wave format (PCM)? It's actually not that bad. The key is to follow the wave format specification



http://replaygain.hydrogenaudio.org/file_format_wav.html
http://ccrma.stanford.edu/courses/422/projects/WaveFormat/

http://www.sonicspot.com/guide/wavefiles.html

CONSTRUCTING THE WAVE
Code for this blog is posted at Github here

So we see that the first 44 bytes of a wave file is dedicated to wave format / header information, from byte 45 onwards contains all the sample data - the bits which make the noise.

In my Wave class - the constructor allows setting of basic Wave settings (sample rate, Resolution [8 or 16 bit], Channel [left, right, mono, left-right stereo]) and creates a byte array 44 cells in size populating it with intial header values. All this code is pretty much standard, not really worth explaining as much of it can be a copy paste job. The main thing is to follow the wave format spec given in the above links.

The sampling code is a bit more interesting as a bit of maths is involved. Essentially DTMF requires the summation of Sine waves of two frequencies to generate a tone recognised by a phone exchange (DUAL TONE multi Frequency). Standard Frequencies for each digit on the phone dial pad (0-9 * # a b c d) can be found at :

http://users.tkk.fi/~then/mytexts/dtmf_generation.html

As far as I am aware, DTMF frequencies are international standards and so the posted frequencies should work with phone exchanges world wide.

In my solution I created a basic data containing class called SineWave. In it's constructor, the frequency (Hz) is given as an int, along with its left and right amplitude (volume) at which the frequency should be should be sampled.

Ok, so looking at the Wave class we have a static method ConstructWave (internal method), which in addition to encoding properties takes an array of SineWaves (the frequencies to be summed) and a TimeSpan (how long the resulting sample should be played).

Say for instance we wanted to generate a tone for the digit '1', using the frequencies specified at http://users.tkk.fi/~then/mytexts/dtmf_generation.html we can construct 2 SineWaves, and pass them to ConstructWave :



//
// playing frequencies at full volume
//
SineWave[] sineWaves = new SineWave[2];
sineWaves[0] = new SineWave(1209, 1, 1);
sineWaves[1] = new SineWave(697, 1, 1);

//
// generate an 8 bit 16kHz sample in mono
//
Wave digitOne = Wave.ConstructWave(sineWaves, 16000, Resolution.EightBit, AudioMode.Mono, TimeSpan.FromMilliseconds(250));


THE IMPLEMENTATION OF SAMPLING

On closer inspection of the ConstructWave method in the Wave class we can see that all sampling is contained in the AppendSample method. Using the target sample rate and sample duration (provided in the Wave constructor) its relatively easy to determine how many bytes the wave sample data should be (Data Size):

i.e.

sample data Byte count = (Sample Rate (Hz) / duration (in seconds)) * no. bytes per sample

WHERE
no. of bytes per sample = (resolution / 8) * no.of channels
IF Mono : no. of channels = 1
ELSE no. of channels = 2



According to the wave header format (above) the total sample data byte count (or data size) should be assigned to bytes 40 - 43. Helper methods ExtractByte and ExtractInt have been written in the Wave class to extract each byte in a 4 byte int (int 32) via bit masking. The Frame Size should also be set in bytes 4 - 7 :


Frame Size = DataSize + 36
[NOTE: 36 is the number of remaining bytes in wave header passed the Frame Size record]


Ok, now to calculate the sample byte data itself..... Both frequencies should be assigned a constant, which in code I have called dataSlice:


dataSlice = (2 * PI) / (waveTime / sampleTime);
[NOTE: waveTime = 1 / frequency (Hz)
sampleTime = 1 / sample rate (Hz)]


Using the number of samples as the loop invariant ( Sample Rate (Hz) / duration (in seconds)) we can calculate each fragment of sample data (bytes 44 - end of array) for the left and right channel (or just the left if mono) as follows:



dataLeft = (Math.Sin (i * FrequencyOneDataSlice) * LeftAmplitude) + (Math.Sin (i * FrequencyTwoDataSlice) * LeftAmplitude) ;

dataRight = (Math.Sin (i * FrequencyOneDataSlice) * RightAmplitude) + (Math.Sin (i * FrequencyTwoDataSlice) * RightAmplitude) ;

WHERE
LeftAmplitude = relative volume of left channel (must be <= 0.5) RightAmplitude = relative volume of right channel (must be <= 0.5) i = current loop iteration



Finally, we mask the resulting number using the resolution of the wave we are sampling (8 / 16 / 24 bit) and store it in the underlying byte array. If you are storing multi channel data (Left, Right, LeftRight Stereo), each data byte should be interleaved....



i.e. 8-bit LeftRight Stereo:

...

waveBytes[n] = ExtractFirstByte(dataLeft)

waveBytes[n+1] = ExtractFirstByte(dataRight)

...


16-bit Stereo:

...

waveBytes[n] = ExtractFirstByte(dataLeft)

waveBytes[n+1] = ExtractSecondByte(dataLeft)

waveBytes[n+2] = ExtractFirstByte(dataRight)

waveBytes[n+3] = ExtractSecondByte(dataRight)

...


And that's it! As the iteration continues, the byte array is filled with Dual Tone byte samples until the target sample size has been reached ( Sample Rate (Hz) / duration (in seconds)).


PLAYING DIRECTLY TO THE SOUNDCARD


Using Pinvoke, we can access winmm.dll - a windows resource to play or save the generated wave as follows:


...

//External method declaration
[DllImport("winmm.dll", SetLastError = true)]
static extern bool PlaySound( IntPtr pszSound, System.UIntPtr hmod, uint fdwSound );

// calling the declared external method
IntPtr ptr = Marshal.UnsafeAddrOfPinnedArrayElement(this.m_waveBytes, 0);
PlaySound(ptr, UIntPtr.Zero, (uint) SoundFlags.SND_MEMORY);
...


References : http://209.171.52.99/audio/concatwavefiles.asp


Wednesday, March 01, 2006

Mp3's & Wave - Constructing DTMF audio files in C#

TERMS:

DTMF: Dual Tone Multi-Frequency - the beeps sent by a phone to the exchange when the User enters a phone number.


THE PROBLEM:

1 - Constructing a wave using a common format (PCM)
2 - Constructing / sampling each digit's tone using standard frequencies
3 - Convert a phone number to a wave
4 - Setting channel (left, right, mono, left - Right stereo) and sampling settings
5 - Integrating unmanaged code into a managed app (using winmm.dll and the Lame encoder)
6 - Encoding the generated wave as an Mp3

** NOTE: This process has been patented as MP3 Telephony by HCV WirelessTM

THE PLATFORM:

C# (easily transferrable to other syntax though) using the Lame Mp3 encoder


BACKGROUND:

I recently had to create a basic DTMF converter for a client using managed code. It's initial inception would be an application with the view that it would eventually be ported over to a website to be used as a service.

Esentially, all the app needed to do was take a phone number string and encode it into its DTMF representation as an Mp3 file. This meant that first I would have to create the sample as a Wave before ripping it as an Mp3 using the Lame encoder. An added feature was to be able to set which channel the DTMF tones would be generated for - Left, Right, Mono, or Left - Right stereo.

Over the next few blog entries I will tackle each segment of the problem.