Counting Records In Large Files The Easy Way

The other day I needed to be able to tell how many records were in some text files. In .NET, this seems pretty easy -- you can just read in the file and count the lines:

  Dim count = File.ReadAllLines(filename).Length

But this can take a long time if the files are really big. And the files I was working with had millions of records.

They also happened to be fixed length records. So all I had to do was find the total size of the file & the length of one record and then divide the former by the latter.

Private Function GetRecordCount(filename As String) As Long
   Dim lineLength = File.ReadLines(filename).First().Length
   lineLength += 2 ' Add 2 bytes for CrLf
   Dim fi = New FileInfo(filename)
   Dim count As Long = fi.Length / lineLength
   Return count
End Function;

Comments

Post a Comment

Popular posts from this blog

Restoring Color Icons in Gimp 2.10

Image
When Gimp 2.10 was released, the new icons for the toolbox items were all black and white. If, like me, you'd like to go back to the color versions, here's how:  Select Edit --> Preferences From the Preferences dialog, under the Interface section, select Icon Theme For the icon theme, select Legacy if you want icons like they were in 2.8.  If you'd like color versions of the new icons, select the icon theme named  Color . 
Read more

PowerBuilder and SQL Server: Tips On Working Together

I recently worked on a project that used PowerBuilder for the front end and SQL Server for the database. I've worked with both of these technologies in the past but this was the first time I used them together. Below are some notes about the things that I learned regarding PB and SQL Server working together. Single Quoted Strings If you do any work with SQL statements in SQL Server Management Studio (SSMS), you'll find that SQL Server only accepts strings if they are surrounded by single quotes. You can use double quotes in the PowerBuilder painters and they'll get translated properly on their way to SQL Server. But you may want to just go ahead and use single quotes everywhere, especially if you are trying to track down a problem by copying SQL from PowerBuilder and pasting it into SSMS. Identity Columns One of my tables had its primary key set in SQL Server to autoincrement. In my datawindow I would insert a row and then do an update. I was expecting the value SQL Server ...
Read more

A tip on multithreading in PowerBuilder

The way you do multithreading in PowerBuilder is to create an object that contains the code you want to run in another thread. You then create an instance of this object as a “shared object”. You also code another object – the “interface object” and pass the shared object a reference to it. You can then call functions on the shared object and have them run in the other thread. The shared object communicates back to the main thread via the interface object. The other day I set up a program using a shared object. I had a shared object that was running some code asynchronously. When it was done, it called a function on the interface object and the interface object called a function in my window to let me know it was complete (by setting a variable). At one point I needed to wait until the shared object was done with its processing before I continued what I was doing in the main thread. I had it coded something like this: Integer li_status Li_status = wf_get_status() Do while li_status = ...
Read more