Archive for the ‘Work’ Category

Long commands on Windows with SCons

January 15, 2015

At my job, where one of my tasks is to handle builds for Windows software using SCons, I recently moved from using SCons 1.2 with some custom modifications to an unmodified SCons 2.3.4. The change comes along with a move to using a newer Visual Studio while still supporting builds with an older one. The newer SCons has trouble with issuing some commands, just like the older version. Neither version can run programs that have whitespace in their path if the entire command is longer than some threshold. The custom modifications I made were to correct the problem, but this time I wanted something less custom and easier to support. Here is a link to the solution I developed; read on for the details.

With shorter commands, SCons issues the commands about the same way it does on Linux, and it works fine. Longer commands run into trouble on Windows; I’m told it has to do with the C runtime libraries. SCons works around this by placing most of a long command into a temporary file and then passing the name of the file preceded by a ‘@’ character as the first command line argument. Microsoft’s compiler and linker will read the temporary file for their arguments.

The implementation of this has a flaw that makes it useless using default installations of Visual Studio. SCons first puts together the command line as though no special long command handling is required. If the result is too long, the command is modified to use a temporary file but is parsed incorrectly; the program to run is taken to be all the characters up to the first whitespace. This makes the command to run “C:\Program”, which isn’t a program, or even an existent path. Everything after the first whitespace is put into the temporary file. It may start with “Files (x86)\Microsoft Visual Studio 10.0\VC\bin\cl.exe”, for example.

I found an attempt at getting SCons to use long commands written by Phil Martin. He provided SCons with a different way to spawn processes. Unfortunately, it doesn’t handle the whitespace issue. By the time the spawn function is called, SCons has already modified the command to use the temporary file. Nevertheless, his implementation was a good place to start. Like his implementation, mine also requires PyWin32.

I modified it to detect the use of a temporary file. When used, the spawn function opens and reads the temporary file to rebuild the complete command line. Then it figures out the whole path to the program, including whitespace, and separates that from the arguments. I also made a modification to produce better error messages from CreateProcess().

Finding the path to the program works best when there is some delimiter in the program path. It is common on Windows to enclose paths that have whitespace with double quotes. The command constructed by SCons using the default program paths lack this delimiter. I solved this by supplying new complete paths for all programs used in Microsoft’s toolchain. It is quite a bother, but I did it the best and most complete way I could figure. This includes creating some build configuration options for several paths, and making two build environments: one for x86 targets and another for amd64 targets.

Hopefully this will help someone else. I really don’t understand why this bug has been around for so long considering that whitespace in paths on Windows is very common.

Advertisements

Windows XP is incompatible with Windows XP

September 25, 2012

I had to go through an excruciatingly lame ethics “course” for my employer that was really a slide show. The only challenge was figuring out which button or link to take next since their interface was horrendous, but that didn’t take long. There weren’t even any questions; I could have ignored all the text of their fit-for-elementary-school tale.

Before I could look at their ethics tale, I had to convince the web server to let me see their crap. In the process, I received a turd that I have tried polishing for you, but the poor fonts and scaling mean you may have to take the link to read it.

System Compatibility Check

System Compatibility Check

One of the computers I use at my job still runs Windows XP. The claim that it isn’t optimized enough shows whoever wrote the error doesn’t know the meaning of optimized. Even better is the suggestion that instead of running Windows XP, I should try running Windows XP. To get this far, I had to use Internet Explorer; the installed Firefox was too new according to the server, but I’m sure it would have worked fine. I tried to print the error page, but IE choked, quit responding to events, and crashed three hours later.

A Backslash is not a Slash

June 26, 2011

I must be playing the crotchety old man today. Maybe it has something to do with not feeling all that well. I’ll get over that, but I might not get over these young’uns who, when I tell ’em to use a slash, ask which one. There is only one. It is used for division. It is used in paths for Internet protocols like HTTP and FTP. It is used as the directory separator on UNIX and UNIX-like systems, which is how it came to be used in Internet protocols. That slash. You know the one.

These young’uns seem to get the slash confused with the backslash. I’m not quite sure why. Try putting in your source code somewhere “a = b \ 2” and I can guarantee that it won’t divide by two unless you have made a conscious effort to do something about that backslash. Try putting backslashes in place of the slashes in the URL bar of your web browser. There is a good chance that the web browser won’t know what you want it to do, and if you get past that, the web server will likely give you a 404.

My best guess is that these young’uns have gotten confused by some Microsoft software allowing both the slash and backslash to be used as directory separators. There was a time when their web browser allowed this and caused some people just learning how to make web sites to make links that were incompatible with anything other than Microsoft software. It seems we have gotten past that bit of confusion from the late 1990’s, yet some software developers get confused when I tell them to use a slash, even though they ought to know them apart to make their division and escape sequences work.

I suppose I might be partly to blame. I regularly use slashes in paths on code that runs on Windows systems because I know that the Windows libraries, like Win32, will interpret the slash as a directory separator. It was a fine decision on Microsoft’s part to assist in porting UNIX software to Windows, and I do use some software at work that started on UNIX and was later ported to Windows. The whole development team does, even if they don’t realize it. So, I use slashes in the paths because they work, and I don’t have to type two characters; using the backslash would start an escape sequence and require another backslash to be interpreted as a single backslash in a string literal.

Still, that doesn’t seem very confusing to me. Software developers need to get this sort of thing straight, so I’m still not clear as to why software developers don’t know what I mean when I say slash. I can understand if they ask about it in the context of a path since not all software on Windows will go both ways, but I get asked outside that context, too. I think when I’m asked which slash to use in the future, I may start answering with use the character for division, or use the character for an escape sequence. If a software developer still doesn’t know what I mean, then they shouldn’t be employed as a software developer. Fortunately, I think it’ll be a while before I find myself working with such an ignorant developer.

Microsoft macros, please see Stroustrup

June 4, 2011

This past week at work, I looked into why I could use a member variable called “interface” just fine in some source files but not others. Preprocessor output showed the name interface was being changed to “struct” in the files that had errors, and sometimes not where I defined the data structure that contained the interface field, but somewhere else later in the file. I eventually tracked down the culprit to objbase.h, a Microsoft supplied header file used by Direct3D 9 and maybe some other stuff. It has this wonderful gem:

#define interface  struct

No C or C++ developer should ever do that. There is no good or justifiable reason to name a macro something that, in many naming conventions, is a perfectly fine name to use for something other than a macro. Most naming conventions use all capital letters only for macro names in order to avoid such confusion.

Bjarne Stroustrup also sees this as a problem. He recommends avoiding macros whenever possible, and I can see why. This is easier to do in C++ with the help of such things as templates that are not in C, but adhering to a good naming convention can eliminate a lot of troubles when macros must be used.

In short, I find it inexcusable for any developer to do something like what I found in objbase.h. Whoever is responsible should be slapped. I can only hope that has already happened.

I/ITSEC 2008

December 4, 2008

20081203-120030ncssI attended I/ITSEC, a trade show focusing on the simulation industry, for a few hours today. Many of the exhibitors were showing military related projects. No one, that I saw, was showing any subsea (underwater) simulations save for CM Labs. That is good for my company since we do subsea simulation, and we use CM Labs’s Vortex physics engine. What they were showing seemed to be more of a demonstration of what Vortex can do rather than a directly competing product.

The most interesting item to me was Concurrent’s Red Hawk Linux, especially their NightStar tool. That tool can show what each processor is running and when along a time line. That is very useful for developing multi-threaded and multi-process systems that need to take advantage of all the processors on modern computers. The demonstration I was given used a system with eight processors (two Intel quad core CPUs) and showed that their software was aware that cores were paired together to share cache memory. Concurrent’s kernel also allows great control over what each processor does and does not run, including allowing or disallowing threads, processes, and interrupts. Very slick and impressive stuff. Makes me want to find that kernel code.


False Steps

The Space Race as it might have been

keithlugcom.wordpress.com/

You Control The Action!

High Frontier

the space colony simulation game

Simple Climate

Straightforwardly explaining climate change, so you can read, react and then get on with your life.