Counting Records In Large Files The Easy Way

The other day I needed to be able to tell how many records were in some text files. In .NET, this seems pretty easy -- you can just read in the file and count the lines:

  Dim count = File.ReadAllLines(filename).Length

But this can take a long time if the files are really big. And the files I was working with had millions of records.

They also happened to be fixed length records. So all I had to do was find the total size of the file & the length of one record and then divide the former by the latter.

Private Function GetRecordCount(filename As String) As Long
   Dim lineLength = File.ReadLines(filename).First().Length
   lineLength += 2 ' Add 2 bytes for CrLf
   Dim fi = New FileInfo(filename)
   Dim count As Long = fi.Length / lineLength
   Return count
End Function;

Comments

Post a Comment

Popular posts from this blog

Restoring Color Icons in Gimp 2.10

Image
When Gimp 2.10 was released, the new icons for the toolbox items were all black and white. If, like me, you'd like to go back to the color versions, here's how:  Select Edit --> Preferences From the Preferences dialog, under the Interface section, select Icon Theme For the icon theme, select Legacy if you want icons like they were in 2.8.  If you'd like color versions of the new icons, select the icon theme named  Color . 
Read more

My Nullable DateTime Picker

Image
I needed to bind a nullable DateTime to a Winforms DateTimePicker control. Unfortunately, the standard DateTimePicker only supports regular DateTime objects. I looked around and found some controls that other people had built, but none of them did exactly what I needed.  I wanted a control where, if the checkbox is unchecked, the value is considered null and the box is blanked out. I couldn’t find one like that, so I built my own. The first one I built was written in VB.Net. In this blog post, we'll create a C# version.  We start by creating a new class and having it inherit from DateTimePicker.  using System.Windows.Forms; namespace NullableSample { public class NullableDateTimePicker : DateTimePicker { } } Then add a nullable DateTime property to hold the binded value.  public class NullableDateTimePicker : DateTimePicker { public DateTime? BindedValue { get; set; } } In the constructor, we set the ShowCheckBox value to true because we want it
Read more

PowerBuilder and SQL Server: Tips On Working Together

I recently worked on a project that used PowerBuilder for the front end and SQL Server for the database. I've worked with both of these technologies in the past but this was the first time I used them together. Below are some notes about the things that I learned regarding PB and SQL Server working together. Single Quoted Strings If you do any work with SQL statements in SQL Server Management Studio (SSMS), you'll find that SQL Server only accepts strings if they are surrounded by single quotes. You can use double quotes in the PowerBuilder painters and they'll get translated properly on their way to SQL Server. But you may want to just go ahead and use single quotes everywhere, especially if you are trying to track down a problem by copying SQL from PowerBuilder and pasting it into SSMS. Identity Columns One of my tables had its primary key set in SQL Server to autoincrement. In my datawindow I would insert a row and then do an update. I was expecting the value SQL Server
Read more