Monday, February 26, 2007

Four Ways to Detect Vista

Pop quiz. How do you detect Windows version? Simple, call the GetVersionEx API.

And now for a bonus question. How do you detect Windows version without a doubt? Well, that's not so simple. Do you disagree? Read on.

Compatibility mode

For some time (I think this functionality appeared in Windows XP), you can execute applications under a compatibility layer. Right-click your exe, select Properties and click on the Compatibility tab. Click on the Run this program in compatibility mode for. Now you can select from the list of many operating system versions - from Windows 95 onwards.

When you do this, plenty of things happen. Firstly, Windows starts faking OS version to your program. Secondly, some API functions work more like they did in the selected OS version.

Why would you do this? Maybe the program in question is not well-behaved and needs this kind of help from the OS. Or maybe it just tests for supported OS version too aggressively and you want to circumvent this test.

In all cases, result is the same - what you get from GetVersionEx is not the true answer. So, I have to repeat my bonus question - how could you detect the true OS version?

Four ways to detect OS

I run into that problem few weeks ago. We have an application that is not yet Vista-ready, mostly because of some DirectX incompatibilities. We don't want negative user experience and therefore we don't allow the app to run on Vista at all. Still, some users may be smart enough to enable compatibility mode for this application, which would make OS version test useless (sadly, compatibility mode doesn't help this application to work correctly on Vista).

I found no good solution but luckily my fellow Slovenian Delphi users did. Even more, they found three alternative solutions.

One is to check for system file that was not present in previous OS versions. For example, one can test for presence of %WINDIR%\System32\ndfetw.dll.

One is to check Notepad.exe version. Under Vista, Notepad has version 6.something while it was 5.something in XP.

And the last one (and the best, in my opinion) is to check if specific API is exported. For example, Vista exports GetLocaleInfoEx from kernel32.dll while previous Windowses didn't.

That's it - now you have one way to get simulated OS version (actually, there are two - you can check the HLKM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\CurrentVersion key) and three to get true OS version.

The last approach is so flexible that we (Miha-R from the Delphi-SI forum, who did most of the work, and me) turned it into a function that can detect all 32-bit OS versions simply by checking for exported API functions. After some work, we created a list of OS-version-determining APIs, all exported from kernel32.dll.

  Vista Server 2003 XP SP1 XP 2000 ME NT 4 98 95 OSR 2 NT 3 95
GetLocaleInfoEx  x                    
GetLargePageMinimum  x  x                  
GetDLLDirectory  x  x  x                
GetNativeSystemInfo  x  x  x  x              
ReplaceFile  x  x  x  x  x            
OpenThread  x  x  x  x  x  x          
GetThreadPriorityBoost  x  x  x  x  x    x        
IsDebuggerPresent  x  x  x  x  x  x  x  x      
GetDiskFreeSpaceEx  x  x  x  x  x  x  x  x  x    
ConnectNamedPipe  x  x  x  x  x          x  
Beep  x  x  x  x  x  x    x    x  x

Writing the function - DSiGetTrueWindowsVersion - was quite simple; just proceed from newest OS to oldest and check for exported APIs. You can get this function (and many more) by downloading the freeware DSiWin32 unit (another effort of Slovenian Delphi community).

Checking for compatibility layer

It would be also interesting to detect whether the application is running under the compatibility layer. There is a registry key that contains this info - Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers, key name is full path of the executable - but the information itself is not reliable.

First, you have to check both HKLM and HKCU branches, as the compatibility may be set by current user and enforced by the system administrator. Second, on 64-bit Windows, you must check the (true) HKLM64 branch, not the (32-bit compatibility) HKLM branch (see the code below for more detail). And third, all this may be to no avail.

The problem lies in a fact that child processes inherit compatibility layer from parents. If you set compatibility mode for executable A and then run executable B from it, executable B won't have any compatibility data set in the registry but nevertheless it will run under the compatibility layer. Still, information in those keys may be of some use sometime.

Demo

Try running it as a normal app and under the compatibility layer.

program IsVista;

{$APPTYPE CONSOLE}

uses
Windows,
SysUtils,
GpVersion,
DSiWin32;

var
testFile: string;

begin
Writeln('GetVersionEx API: ', CDSiWindowsVersionStr[DSiGetWindowsVersion]);
Writeln('HKLM CurrentVersion registry value: ', DSiReadRegistry(
'\SOFTWARE\Microsoft\Windows NT\CurrentVersion', 'CurrentVersion', '',
HKEY_LOCAL_MACHINE));
Writeln('HKLM ProductName registry value: ', DSiReadRegistry(
'\SOFTWARE\Microsoft\Windows NT\CurrentVersion', 'ProductName', '',
HKEY_LOCAL_MACHINE));
testFile := IncludeTrailingPathDelimiter(DSiGetSystemFolder) + 'ndfetw.dll';
Writeln('Presence of file ', testFile, ': ', BoolToStr(FileExists(testFile), true));
Writeln('Notepad.exe version: ',
CreateResourceVersionInfo(IncludeTrailingPathDelimiter(DSiGetWindowsFolder) +
'notepad.exe').GetFormattedVersion(verFullDotted));
Writeln('Presence of GetLocaleInfoEx API: ',
BoolToStr(GetProcAddress(GetModuleHandle('kernel32'), 'GetLocaleInfoEx') <> nil, true));
//http://msdn2.microsoft.com/en-us/library/aa480152.aspx
Writeln('HKCU AppCompatFlags for ', ParamStr(0),': ',
DSiReadRegistry('Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers',
ParamStr(0), '', HKEY_CURRENT_USER));
Writeln('HKLM AppCompatFlags for ', ParamStr(0),': ',
DSiReadRegistry('Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers',
ParamStr(0), '', HKEY_LOCAL_MACHINE));
if not DSiIsWow64 then
Writeln('Not a 64-bit OS, KEY_WOW64_64KEY not tested')
else begin
//http://msdn.microsoft.com/library/en-us/sysinfo/base/about_the_registry.asp
Writeln('HKLM64 CurrentVersion registry value: ', DSiReadRegistry(
'\SOFTWARE\Microsoft\Windows NT\CurrentVersion', 'CurrentVersion', '',
HKEY_LOCAL_MACHINE, KEY_QUERY_VALUE or KEY_WOW64_64KEY));
Writeln('HKLM64 ProductName registry value: ', DSiReadRegistry(
'\SOFTWARE\Microsoft\Windows NT\CurrentVersion', 'ProductName', '',
HKEY_LOCAL_MACHINE, KEY_QUERY_VALUE or KEY_WOW64_64KEY));
Writeln('HKLM64 AppCompatFlags for ', ParamStr(0),': ',
DSiReadRegistry('Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers',
ParamStr(0), '', HKEY_LOCAL_MACHINE, KEY_QUERY_VALUE or KEY_WOW64_64KEY));
end;
Writeln('DSiGetAppCompatFlags: ', DSiGetAppCompatFlags(ParamStr(0)));
Writeln('DSiGetTrueWindowsVersion: ', CDSiWindowsVersionStr[DSiGetTrueWindowsVersion]);
Readln;
end.

Friday, February 23, 2007

Nasty inline codegen bug in BDS 2006

I'm using assign-test-and-do-something code pattern quite frequently. You know, code like this:

Result := CalculateSomething(some, param); // assign
if not Result then // test
Exit; // do something

As all real programmers, I'm lazy and tend to rewrite such code as:

if Asgn(Result, CalculateSomething(some, param)) then
Exit;

Most of the dirty work is done by the Asgn function, which acts as a splitter - it copies one value to two destinations - output parameter and function result.

function Asgn(var output: integer; const value: integer): integer;
begin
output := value;
Result := value;
end;

(Actually, I have four overloaded Asgn functions, but that's beside the point.)


Yesterday, I did two things almost at the same time (which was, in hindsight, very lucky - it helped me to find the problem in "mere" 30 minutes). Firstly, I changed all Asgn functions to be inlined on modern Delphis and secondly, I used same variable as part of the value calculation and as the output parameter. In other words, I did this: 

function Asgn(var output: integer; const value: integer): integer; inline;
begin
output := value;
Result := value;
end;

copy := Asgn(v1, v1+v2);

(It was slightly more complicated than that but the code above nicely summarizes the problem.)


This code should calculate a sum of two variables and assign it to both v1 and copy. It should, I said, but it didn't. Correct value was assigned to v1, but something completely different was assigned to the copy variable. When I removed the inline, Delphi immediately started to generate correct code.


So, for now, I removed inline from all my Asgn functions. I also know that I'll be extremely causious when using inline in the future.


CodeGear has been notified of the problem (QC #41166) and I hope they'll be able to fix the codegen before Delphi 2007 is out.


If you don't believe me, try it for yourself. Demo is below. It should output four numbers 17, but it doesn't.

program Project8;

{$APPTYPE CONSOLE}

uses
SysUtils;

function Asgn1(var output: integer; const value: integer): integer;
begin
output := value;
Result := value;
end;

function Asgn2(var output: integer; const value: integer): integer; inline;
begin
output := value;
Result := value;
end;

var
v1, v2: integer;

begin
v1 := 10;
v2 := 7;
Writeln(Asgn1(v1, v1+v2));
Writeln(v1);
v1 := 10;
v2 := 7;
Writeln(Asgn2(v1, v1+v2));
Writeln(v1);
Readln;
end.

Thursday, February 08, 2007

Free tools I use for development

Now and then I find a nice free tool hidden on the web. Lately, I found some nice gems mentions on other blogs and I thought I should share my list with you, dear reader. Maybe you'll find something of use and maybe you could recommend some other good tool to me.

 

Another Lens  - A screen magnification tool, written in Delphi 4. [Bjorn Ischo; link from the author's home is not working, this is a correct one: http://www.b-zone.de/zips/lens.zip]

Desktop Icons Save and Restore* - Shell extension that allows you to save and restore position of icons on the desktop. [Jamie O'Connell, http://www.midiox.com/html/desktop.htm]

FireBird - Great SQL database server. I'm particularly interested in the embedded version. [Firebird Foundation, http://www.firebirdsql.org/, http://www.ibphoenix.com/]

GpProfile - My old but still useful profiler. (I am working on a new version.) [Primoz Gabrijelcic, http://gp.17slon.com/gpprofile/index.htm]

GraphEdit -  A tool for visual DirectX filter graph editing.  [Microsoft, part of the DirectX SDK, http://msdn2.microsoft.com/en-us/library/ms783346.aspx, but is easy to find on the web if you don't want to download and install entire SDK]

IcoFX - Great icon editor and newest addition to my toolbox. [Attila Kovrig, http://icofx.xhost.ro/]

Microsoft Spy++ - Window browser. [Microsoft, part of Visual Studio, http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcug98/html/_asug_home_page.3a_.spy.2b2b.asp, easy to find on the web if you don't need complete Visual Studio]

Putty - Ssh and sftp client. [Simon Tatham, http://www.chiark.greenend.org.uk/~sgtatham/putty/]

IBExpert Personal Edition - InterBase/FireBird management tool. [HK Software, http://www.hk-software.net/ibexpert.com/content/doc_122943-5-10-00.php]

Synergy* - Control multiple computers with one mouse and keyboard. [http://synergy2.sourceforge.net/authors.html, http://synergy2.sourceforge.net/, http://www.stevetrefethen.com/blog/default,month,2006-06.aspx]

TightVNC* - Remote control client and server. [http://www.tightvnc.com/people.html, http://www.tightvnc.com/]

VLC Media player - Media player, which is of particular interest to me because it can play (and wrap/unwrap) DVB Transport Streams. [VideoLAN, http://www.videolan.org/]

WinCVS - CVS GUI which is so good that I don't want to switch to Subversion. [http://sourceforge.net/project/memberlist.php?group_id=10072, http://sourceforge.net/projects/cvsgui]

Window Clippings - Nice screen grabber, useful when writing documentation. [Kenny Kerr, http://weblogs.asp.net/kennykerr/archive/2005/09/30/Window-Clippings-1.0.aspx]

Windows SysInternals tools - I need them all. Constantly. [Mark Russinovich, http://www.microsoft.com/technet/sysinternals/default.mspx]

Wink - Records all changes on the computer screen and makes a movie. Good for tutorials and to demonstrate a problem to a coworker. [DebugMode, http://www.debugmode.com/wink/]

WinPCap - Tool to monitor and transmit data in link-layer network access, used in Wireshark (below). [http://www.winpcap.org/misc/credits.htm, http://www.winpcap.org/]

WinSCP - Graphical sftp interface, based on the Putty. [http://winscp.net/eng/docs/contributions, http://winscp.net/eng/index.php]

Wireshark [ex Ethereal] - Capture and analyze network packets. [http://www.wireshark.org/about.html, http://www.wireshark.org]

XanaNews - Good newsgroups reader, written in Delphi. [Colin Wilson, http://www.wilsonc.demon.co.uk/xananews.htm]

XN Resource Editor - Resource editor, written by the XanaNews author. Includes simple icon editor. [Colin Wilson, http://www.wilsonc.demon.co.uk/d10resourceeditor.htm]

 


*More on this product: Remote access to the fourth power

Friday, February 02, 2007

When Vista doesn't print

I installed Vista RTM immediately as it was available on the MSDN.

That was quite a surprise for me as I thought that many months would pass before my first Vista installation. But alas, I had to write an article on Vista for Slovenian leading computing magazine Monitor and I just got completely new Vista-ready laptop which I could use for playground. (It got Windows XP Home preinstalled - yuck!)

Anyway, I installed Vista (or, to be more precise, Windows Vista Ultimate) on that laptop. Everything worked fine (even though I had to scrap together few drivers from all over the internet). Everything except that I couldn't print. I have this nice colour laser printer (Samsung CLP-510, really smooth) in the office, and it is connected to the Windows 2003 Server. All other computers (running XP everyone of them) could use this shared printer but not the Vista, nooo sir.

It was quite an interesting malfunction, too - when I tried to connect to my shared printer, Vista started copying drivers (XP drivers, of course), and then reported that driver installation 'ran out of memory'. OK, I only have measly 2 gigs installed, but still...

I tried a different trick - installed printer driver locally (no problem here) and only after that tried to connect to the shared printer (in hope that currently installed driver will be used). But nooo, Vista reported same error again.

I gave up then saying: "Oh, well, it's only about two months until Vista is available for general public. Surely Samsung will have Vista drivers ready by then." But the problem was still nagging me and I spilled out my soul to my friends on Slovenian Delphi forum. Somebody has an interesting idea - why don't I connect to the Samsung as if it were a TCP/IP printer and then spool EMF content to the server. Still, he had no idea how to make this work in practice.

I searched for the solution occasionally but found out that Microsoft documentation on that topic is really hard to find if you don't know what you're searching for. I did'nt spend much time on that, though, as I could always use the printer from my main computer.

Solution, at last!

Today I returned to the problem and somehow managed to find the answer in Microsoft's documentation. It really is simple - when you already know how to do it :( .

For anybody trying to solve a similar problem - and for all of you that just want to learn more - here is the solution.

First, install Print Services for Unix on the Windows 2003 Server (Control Panel, Add or Remove Programs, Add/Remove Windows Components, Other Network File and Print Services, Details). Make sure that the TCP/IP Print Server service is running.

Next, install LPR Port Monitor feature on the Windows Vista (Control Panel, Programs and Features, Turn Windows features on or off, Print Services).

Create new port on the Windows Vista computer. Run Printers and clik Add Printer. Choose Add a local printer. Select Create a new port setting and set Type of port to LPR Port.

Click Next and enter address of the server providing Unix printing services and name of the printer.

Click OK and proceed as usual - select printer manufacturer etc.

All work to no avail?

Using this procedure I managed to get my Samsung working from the Vista.

I learned a lot and maybe I will be able to use this knowledge in the future.

Maybe it will help somebody out there with a similar (or completely different) problem.

It didn’t help me at all :(

You see, it is February already and Samsung published updated printer drivers two days ago.

Tuesday, January 30, 2007

Microsoft Keyboard Layout Creator 1.4 has been released

If you need it, download it from here.

If you don't know it - MKLC is a great tool that will help you create customized keyboards (with some changes or completely scrambled layout) in minutes. It's a great programming tool also - you can make sure that bot important special characters (i.e. []{}) and national characters are easily accessible.

[Source: Sorting It All Out]

Friday, January 26, 2007

Fast binary searching, part 2: Hard data

This is an update on my Faster than the speed of binary search entry. Maybe it would be good to refresh your memory before proceeding, brave reader ...

Last time I promised some hard data for multi-GB files. Finally, I had some time to do the measurements. I was doing the usual space & time profiling (we are talking about 30 - 60 minute execution times here so each 10% helps a lot) and this was a natural extension of tests.

Sadly (from the viewpoint of a perspective author that introduced the improved binary search algorithm into the application) I have redesigned the most time-consuming part of our app and the biggest list (requiring most lookups) is not used anymore. Therefore, numbers are not so large as I expected three months ago. Data is still interesting, especially as the analysis exposed two anomalous cases.

Let's first take a look at a 'normal' case. The list had between 1.596,266 and 1.803,027 entries (lists are live and in this case entries were removed from the list during the execution) and program executed 171,191 searches. Standard binary search needed 20.8 rounds (bisection steps) to resolve search query on average while my 'bias' version needed only 6.7 rounds. A more detailed distribution of number of rounds is shown in the following graph.

standard vs. biased search

All good and well then, but this was exactly the situation for which the algorithm was designed. Let's take a look at the first anomalous case, where biased search exhibits really bad behaviour.

anomalus behaviour, bad biased search

As you can see, the list had from 0 to 37,300 entries (0 at beginning) and there were 37,301 searches. In fact, each addition to the list was followed by a search request which always found the result somewhere in last 30 entries. Standard binary search needed 13.9 rounds on average while biased version needed 13.5 rounds, which is almost the same (and is in fact slower due to the slower inner loop of the biased version).

The distribution of rounds in biased search is also weird. It must either be a bug in the algorithm or a weird data pattern. It turned out to be the latter. In this case, the list contained data in clusters with big differences between each cluster. Something like this list with 40 entries: 1, 2, 3 ... 20, 1.000,000 1.000,001 .. 1.000,019 (notice the big jump between 20 and 1.000,000). Even worse, the code tries to find smallest number (or a number very close to it) in last cluster (1.000,000 in this case). My biased version first sets lower and upper bound of the search and then calculates the bias (1000000 - 1)/(1000019 - 1), which is in this case 0.999981 or almost 1 [see the code in previous article]. That causes selected 'midpoint' to be equal to the higher bound - 1. Search loop is then repeated, bias is almost the same and upper bound is again reduced by 1. This repeats until upper bound reaches the element we are searching for.

As you've seen, the search degrades to a simple linear search (with complicated calculations behind) in this case. Which is good when we want to find the last element (that happens when last cluster has only one element) or a element very much near the end of the list (when last cluster is small), but degrades quite badly when last cluster is large.

The second anomalous case is actually a special case of the first one, but in this case the cluster was always just one element big. In other words, the code always searched for the largest value in the list. In this case, biased search almost always retrieved the correct number in the first attempt, which is significantly faster than the standard binary search (16.5 rounds on average).

anomalus behaviour, good biased search

So what can we do to improve the situation? One way is to add a special search mode for my 'weird' data, but I really dislike this idea, as I don't know what kind of data will be passed to my (fairly generic) classes in the future.

Another idea was to limit the maximum bias. In standard binary search, the bias is always 0.5 (midpoint between low and high bound) while in the problematic case the bias is almost 1. If we limit the maximum bias to, say, 0.9, the code never degenerates into linear search.

When such clamping is applied, first anomalous case behaves much better (average number of rounds has been reduced to 8.2).

anomalus behaviour, bad biased search, clipping

That's fine, but how would that change affect the other two cases? The 'normal' case behaves worse than before (average number of rounds went from 6.7 to 9.6).

standard vs. biased vs. clipped biased search

That could be acceptable, but in the second anomalous case the number of rounds increases almost by factor 6.

anomalus behaviour, good biased search, clipped

It's clear that this simple solution indeed improves the worst case but degrades all other searches by a noticeable amount.

I had other ideas on how to improve the search algorithm but at the end I left it as it is now (the 'bias' version). After all, in the worst case biased search still makes less rounds than standard search; in the normal case, biased search works as expected and in the best case, biased search works much much better than the standard search.

I hope that I made my point clear - always, but really always analyse the program behaviour on real data, not on test cases.

The other point that is not so obvious is - Excel 2007 charting capabilities rock :)

 

Thursday, January 25, 2007

There is no good place to advertise Delphi blog ...

Craig Stunz kindly noticed my blog (thanks, Craig) and in private discussion (because commenting on teamb.com blogs still doesn't work) we talked about possibilities to tell about your new blog to other Delphi programmers.

Few possibilities appeared - nontech newsgroup, DelphiFeeds aggregator (my ideas), community homepage and 'Get Published' on CDN (his ideas) but they are all flawed - many developers don't read any of those ...

What we really need is a Delphi Wiki, maintained by CodeGear and linked from the welcome page. Plus a RSS for this wiki so that one can follow updates without too much trouble ... (Yes I know that there is already a Delphi Wiki but I don't think it has large audience. And the blog list there is quite short.)

CodeGear, if anybody is listening ... could something like this happen?

Using interfaces for multithreading synchronisation

This seems to be an interesting idea so I'm noting it down for possible future work on the subject.

 

Technorati tags: , ,

Recovering from Windows Update errors

Mostly for my own backup - here are two links to articles on how to make misbehaving Windows Update work again.

Michael Kaplan: Maybe your computer doesn't like you?

Scott Hanselman: Resetting Microsoft Update - Error 0x8024001D

 

Technorati tags: ,

Tuesday, January 23, 2007

Viewing composite files in Total Commander

[This is a third and probably last article in the 'embedded file system' series. If you want to refresh your memory or if you're new to my blog, read previous two installments here and here. You may also want to refresh your copy of the GpStructuredStorage unit as it was updated after the last article in the series was produced.]

I'm a great fan of Total Commander. I've been using it for who-knows-how long - I started in the Windows 3.1 times, when it was still called Windows Commander. Even before, in the DOS, I was using Norton Commander. You'd never guess, huh? ;)

One of the thing I like about Total Commander is its ability to work with various packed formats as if they were a part of a file system. You just press Enter and ZIP files opens as if it were a normal folder, and you can use standard Copy, Move, Rename etc commands on files inside the archive. Great stuff.

Another thing I like about Total Commander is its extensibility. The author has documented four types of extension interfaces (to access packed files, foreign file systems, display new file formats and to extract data from files). You have to write a DLL exporting few simple (or not) functions and plunk it into the TC plugins folder. And it "just works" after that.

The third thing I like about Total Commander is the fact that header files for plugins are all available in .h and .pas format, making Delphi implementation a snap :)

File system or packed file?

For quite some time I couldn't decide whether I should implement support for GpStructureStorage composite files as a files system plugin or as a packed file (archive) plugin. At the end, two things prevailed - packed file plugins are simpler to write and they are simpler to use (in the the Total Commander).

I started by browsing the WCX packer plugins documentation (WCX is extension for packer plugin DLLs). As WCX is simple DLL with only stdcall functions, there are no problems implementing it in Delphi. Great. I also found out that I don't have to implement much as all plugin functions are access dynamically and I can just ignore the ones I don't need. Even better!

It turns out that there are three classes of functions - for reading, writing, and packing in memory (as opposed to packing directly to the file). I only had to implement the first batch of functions as I only need read access to composite files. (Yes, writing to composite files would sometime be nice, too, but I haven't had time to implement it. You are welcom to improve my packer in that direction, if you have both time and knowledge.)

The WCX documentation told me that I only have to implement six functions:

function  OpenArchive(var ArchiveData: TOpenArchiveData): THandle; stdcall;
function ReadHeader(hArcData: THandle; var HeaderData: THeaderData): integer; stdcall;
function ProcessFile(hArcData: THandle; Operation: integer; DestPath, DestName: PChar): integer; stdcall;
function CloseArchive(hArcData: THandle): integer; stdcall;
procedure SetChangeVolProc(hArcData: THandle; pChangeVolProc: TChangeVolProc); stdcall;
procedure SetProcessDataProc(hArcData: THandle; pProcessDataProc: TProcessDataProc); stdcall;

I also choose to implement two additional functions:

function  GetPackerCaps: integer; stdcall;
function CanYouHandleThisFile(FileName: PChar): boolean; stdcall;

GetPackerCaps is really simple - it informs the Total Commander of your plugin capabilities. I specified options PK_CAPS_BY_CONTENT (TC should detect composite files by content, not by extension) and PK_CAPS_SEARCHTEXT (searching inside composite files is allowed).

function GetPackerCaps: integer;
begin
Result := PK_CAPS_BY_CONTENT + PK_CAPS_SEARCHTEXT;
end; { GetPackerCaps }

The tricky part here is that this function is called only when plugin is installed. If you later change plugin capabilities and recompile, new capabilities won't be detected! You have to remove the plugin from TC and reinstall it so that GetPackerCaps is called again.


If there was a standard extensions for GpStructuredStorage composite files, I could just register it with Total Commander and skip the CanYouHandleThisFile function. As there is no such extension, I had to write it, but it was really simple as GpStructuredStorage already contains a function to check for valid composite file:

function CanYouHandleThisFile(FileName: PChar): boolean; stdcall;
begin
Result := CreateStructuredStorage.IsStructuredStorage(FileName);
end; { CanYouHandleThisFile }

Now I can just select any composite file and press Enter or Ctrl+PgDn to open it inside TC. Of course, I still need to implement few functions to actually read the composite file ...


Implementing read access


To allow Total Commander to list contents of a composite file, we need to implement OpenArchive, ReadHeader, ProcessFile, and CloseArchive functions. An example from the WCX plugin documentation shows how they are used in TC to access the archive contents:



OpenArchive()          with OpenMode==PK_OM_LIST
repeat
   ReadHeader()
   ProcessFile(...,PK_SKIP,...)
until error returned
CloseArchive()


OpenArchive must return unique handle to the archive - an opaque object that is passed to ReadHeader, ProcessFile, and CloseArchive so that they know what archive they are working on. In theory, we could return IGpStructuredStorage instance converted to a THandle, but there is a problem with this - TC expects the contents of an archive to be a simple list of files and we have directory structure stored inside composite files.


Somehow, we have to walk the directory structure inside the composite file and convert it into a flat list. This could, for example, be done in the OpenArchive call. Still, we have to keep this list and the IGpStructuredStorage interface as, maybe, we will have to extract data from it.


If user copies a file from an archive, TC executes similar loop to the one above:



OpenArchive()          with OpenMode==PK_OM_EXTRACT
repeat
   ReadHeader()
   if WantToExtractThisFile()
      ProcessFile(...,PK_EXTRACT,...)
   else
      ProcessFile(...,PK_SKIP,...)
until error returned
CloseArchive()


Because some archives may not have directory and direct access to files (tar comes to mind), TC reads header of each file and then either skips or extracts it.


Therefore I have created a global list containing TGpStructuredStorageManipulator objects, each holding a reference to a IGpStructuredStorage interface plus an internal state required for the file enumeration. WCX DLL functions are just mappers into global list and manipulator objects:

function OpenArchive(var ArchiveData: TOpenArchiveData): THandle;
begin
Result := GStructuredStorageList.OpenArchive(ArchiveData.ArcName, ArchiveData.OpenResult);
ArchiveData.CmtBuf := nil;
ArchiveData.CmtBufSize := 0;
ArchiveData.CmtSize := 0;
ArchiveData.CmtState := 0;
end; { OpenArchive }
function ReadHeader(hArcData: THandle; var HeaderData: THeaderData): integer; stdcall;
var
manip: TGpStructuredStorageManipulator;
begin
manip := GStructuredStorageList.LocateHandle(hArcData);
if not assigned(manip) then
Result := E_NOT_SUPPORTED
else if manip.GetNextHeader(HeaderData) then
Result := 0
else
Result := E_END_ARCHIVE;
end; { ReadHeader }
function ProcessFile(hArcData: THandle; Operation: integer; DestPath, DestName: PChar):
integer; stdcall;
var
manip : TGpStructuredStorageManipulator;
outFile: TFileStream;
begin
if (Operation = PK_SKIP) or (Operation = PK_TEST) then
Result := 0
else begin
manip := GStructuredStorageList.LocateHandle(hArcData);
if not assigned(manip) then
Result := E_NOT_SUPPORTED
else begin
try
outFile := TFileStream.Create(string(DestPath) + string(DestName), fmCreate);
try
Result := manip.ExtractCurrentTo(outFile);
finally FreeAndNil(outFile); end;
except
on E: EFCreateError do
Result := E_ECREATE;
end;
end;
end;
end; { ProcessFile } 
function CloseArchive(hArcData: THandle): integer; stdcall;
begin
Result := GStructuredStorageList.CloseArchive(hArcData);
end; { CloseArchive }

If you'll be browsing the code, you'll notice that I decided against creating a long list of all files and folders inside the composite file in the OpenArchive procedure. Instead, I'm keeping internal enumerator state in the TGpStructuredStorageManipulator object and GetNextHeader method of this class uses this internal state to walk over the composite file. It seemed neater that way.


Oh, and what about SetChangeVolProc and SetProcessDataProc? They just set callback functions that we don't need. Read more on them in the WCX plugin documentation.


WRITING TO COMPOSITE FILES


Adding write and delete access to the plugin should be easy - one only has to implement PackFiles and DeleteFiles. Implementation should be quite straightforward as theay don't use the 'iterate until you find the correct file' approach. Instead of that, TC sends full file name (with folders and everything) to the procedure. But as I had no need for write access, I haven't implemented that part yet. Feel free to do it instead of me (and don't forget to update GetPackerCaps accordingly :) ).

Tuesday, January 09, 2007

No Delphi content here, just woodworking ...

No, I didn't forget that I'm writing a blog. Yes, it was a long time since I wrote anything here. I am really sorry for that. I'm just fixing thousand things all at once in an important part of our software (and I intend to blog about some aspects of that - as soon as I can find a little time to prepare the article).

In the meantime, I'd like to show you a little of what I do in the free time - make stuff from the wood. No real photos today (for the curious, there is a gallery on my web site), just two SketchUp models.

First a table I've been building for my wife's sister and her family. It will be made of ash and is currently in a preparation phase - half of the wood have been planned and cut to size and other half is still in a raw form. Dimensions are 100 x 80 x 35 cm and I think it will weight around 50 kg (yep, lots of solid wood here).

Another project got stuck in the designing phase - my 'customer' (I only work for friends and family, I'm not selling products on the open market) decided that she will continue to live without a coffee table.

It ought to be a simple wood-and-glass table, looking roughly like this (all rough edges are courtesy of my inability to draw nice curves with SketchUp).

If somebody wants a deeper look, I can provide the SketchUp source files. 

Technorati tags:

Friday, December 01, 2006

Installing BDS Update 2 on Vista

Few days ago I had an opportunity to test BDS 2006 on a Vista RTM (I was writing an article on Vista for the Monitor magazine and I did some personal tests when all other work was finished). Installing BDS was not a problem, but Update 2 was stubbornly refusing to find the BDS installation. All I was getting was this error:

Setup cannot continue. This is Update 2 setup for Borland Developer
Studio 2006 Architect Edition. Please cancel this and install 
Edition of Update 2

(Sorry, no picture - I forgot to take a screen shot at that time.) There was a double space between 'install' and 'Edition' so obviously something went very wrong with the installer and it didn't know anymore what it's supposed to do.

Soon  I found out that I'm not the only one with the problem. I also found a workaround on Borland support site. As I was already very skeptical about the whole UAC thing, I switched it off without a thought.

That's how you do it: Click the Start button, type user in the search box and run User Accounts applet. Click the Turn User Account Control on or off link. Uncheck the User User Account Control (UAC) to help protect your computer. Click OK. Be aware that restart will be required to disable this setting.

After installing BDS 2006 Update 2 you can turn UAC on with the similar procedure - but that's up to you. I left it disabled. Hate the damn thing.

To return to my installation problems - this workaround didn't work for me. I was not getting that error anymore but it got even worse - Update 2 just closed and Vista reported something like "Microsoft Installer found a problem and cannot continue". I tried few times )(stubborn programmer) but the result was always the same. I even rebooted and retried and the I got lucky - Update 2 still closed but Vista displayed an error hint in the taskbar saying that my app got closed due to a DEP protection. (Yes, I was testing Vista on AMD 64 processor, which has hardware DEP support built in. No, I was running Vista 32-bit.)

That's something, but how do we disable DEP? Vista, like Windows XP, only has two options - enable DEP for all system programs (default) or enable DEP for all programs except the one in the exclusion list. (DEP settings are hidden in System Properties, Performance Options, in case you can't find them.)

I searched the MSDN and found out that I have to disable DEP during boot. Usually (since NT times) one would have to edit c:\boot.ini file. Surprise, surprise, this file is gone in Vista! To edit boot configurations, you have to use a command-line tool bcdedit. Documentation on it is not very well organized. The best starting point is probably Boot Options for Driver Testing and Debugging.

It turns out it's easiest to create new configuration with the /copy switch:

C:\Users\gabr>bcdedit /copy {current} /d "DEP disabled"
The entry was successfully copied to {69ef4ad0-8123-11db-aa33-0015c53ad3ed}.

I based the new configuration on the current configuration (which is the simplest approach to creating a new configuration). Bcdedit returned GUID of the new configuration. We'll need this GUID to modify the new configuration:

C:\Users\gabr>bcdedit /set {69ef4ad0-8123-11db-aa33-0015c53ad3ed} nx AlwaysOff
The operation completed successfully.

The 'nx AlwaysOff' setting disables the execution prevention (see BCDEdit /set for more parameters).

We can verify that configuration really got created:

Windows Boot Manager
--------------------
identifier              {bootmgr}
device                  partition=C:
description             Windows Boot Manager
locale                  en-US
inherit                 {globalsettings}
default                 {current}
resumeobject            {53a24ff1-7f3d-11db-9cd4-a2298b078311}
displayorder            {current}
                        {69ef4ad0-8123-11db-aa33-0015c53ad3ed}
toolsdisplayorder       {memdiag}
timeout                 30

Windows Boot Loader
-------------------
identifier              {current}
device                  partition=C:
path                    \Windows\system32\winload.exe
description             Microsoft Windows Vista
locale                  en-US
inherit                 {bootloadersettings}
osdevice                partition=C:
systemroot              \Windows
resumeobject            {53a24ff1-7f3d-11db-9cd4-a2298b078311}
nx                      OptIn

Windows Boot Loader
-------------------
identifier              {69ef4ad0-8123-11db-aa33-0015c53ad3ed}
device                  partition=C:
path                    \Windows\system32\winload.exe
description             DEP disabled
locale                  en-US
inherit                 {bootloadersettings}
osdevice                partition=C:
systemroot              \Windows
resumeobject            {53a24ff1-7f3d-11db-9cd4-a2298b078311}
nx                      AlwaysOff

 Alternatively, you can check the System Properties, Startup and Recovery.

 

(I apologize for the quality of the picture - that's purely Windows Liver Writer's doing.)

If you made those changes and reboot the system, it will display the usual NT style 'select your configuration' prompt. I did that, booted without a DEP and finally I could install Update 2. I rebooted back with DEP enabled and BDS 2006 works just fine.

Sadly, I had to return the test computer next day so now I'm running Vista only on my Dell notebook. At least BDS works fine here, too.

Wednesday, November 15, 2006

Bulk update

I have updated few of my BSD-licensed units. Enjoy.

GpLists 1.22a

GpHugeFile 4.0b

GpStructuredStorage 1.11

GpSync 1.19a

  • Raise exception if caller tries to post a message that is larger than the message queue.

GpTextFile 4.0a

  • Bug fixed [found by Brian D. Ogilvie]: When cfUseLF was set, /CR/ was used as line delimiter in Writeln (/LF/ should be used).

GpTextStream 1.04a

  • Bug fixed: When cfUseLF was set, /CR/ was used as line delimiter in Writeln (/LF/ should be used).

GpTimezone 1.21b

  • Modified UTCTo[TZ]LocalTime/[TZ]LocalTimeToUTC functions to automatically execute FixDT on the result.