Posts Tagged ‘windows’

Long commands on Windows with SCons

January 15, 2015

At my job, where one of my tasks is to handle builds for Windows software using SCons, I recently moved from using SCons 1.2 with some custom modifications to an unmodified SCons 2.3.4. The change comes along with a move to using a newer Visual Studio while still supporting builds with an older one. The newer SCons has trouble with issuing some commands, just like the older version. Neither version can run programs that have whitespace in their path if the entire command is longer than some threshold. The custom modifications I made were to correct the problem, but this time I wanted something less custom and easier to support. Here is a link to the solution I developed; read on for the details.

With shorter commands, SCons issues the commands about the same way it does on Linux, and it works fine. Longer commands run into trouble on Windows; I’m told it has to do with the C runtime libraries. SCons works around this by placing most of a long command into a temporary file and then passing the name of the file preceded by a ‘@’ character as the first command line argument. Microsoft’s compiler and linker will read the temporary file for their arguments.

The implementation of this has a flaw that makes it useless using default installations of Visual Studio. SCons first puts together the command line as though no special long command handling is required. If the result is too long, the command is modified to use a temporary file but is parsed incorrectly; the program to run is taken to be all the characters up to the first whitespace. This makes the command to run “C:\Program”, which isn’t a program, or even an existent path. Everything after the first whitespace is put into the temporary file. It may start with “Files (x86)\Microsoft Visual Studio 10.0\VC\bin\cl.exe”, for example.

I found an attempt at getting SCons to use long commands written by Phil Martin. He provided SCons with a different way to spawn processes. Unfortunately, it doesn’t handle the whitespace issue. By the time the spawn function is called, SCons has already modified the command to use the temporary file. Nevertheless, his implementation was a good place to start. Like his implementation, mine also requires PyWin32.

I modified it to detect the use of a temporary file. When used, the spawn function opens and reads the temporary file to rebuild the complete command line. Then it figures out the whole path to the program, including whitespace, and separates that from the arguments. I also made a modification to produce better error messages from CreateProcess().

Finding the path to the program works best when there is some delimiter in the program path. It is common on Windows to enclose paths that have whitespace with double quotes. The command constructed by SCons using the default program paths lack this delimiter. I solved this by supplying new complete paths for all programs used in Microsoft’s toolchain. It is quite a bother, but I did it the best and most complete way I could figure. This includes creating some build configuration options for several paths, and making two build environments: one for x86 targets and another for amd64 targets.

Hopefully this will help someone else. I really don’t understand why this bug has been around for so long considering that whitespace in paths on Windows is very common.

The Windows search dog is high

October 19, 2011

The Windows search dog is high again.

The search dog is high

That stupid dog can’t count and got the path formats confused. The last two results look like the first two with relative paths fit for a UNIX-like machine. I had this happen a few times while purging the computer I use at work of useless files. Several would show up all UNIX-y with slashes, and when I requested all the files be deleted, I was told that the files just found didn’t exist.

That dog is so very high.

A Backslash is not a Slash

June 26, 2011

I must be playing the crotchety old man today. Maybe it has something to do with not feeling all that well. I’ll get over that, but I might not get over these young’uns who, when I tell ’em to use a slash, ask which one. There is only one. It is used for division. It is used in paths for Internet protocols like HTTP and FTP. It is used as the directory separator on UNIX and UNIX-like systems, which is how it came to be used in Internet protocols. That slash. You know the one.

These young’uns seem to get the slash confused with the backslash. I’m not quite sure why. Try putting in your source code somewhere “a = b \ 2” and I can guarantee that it won’t divide by two unless you have made a conscious effort to do something about that backslash. Try putting backslashes in place of the slashes in the URL bar of your web browser. There is a good chance that the web browser won’t know what you want it to do, and if you get past that, the web server will likely give you a 404.

My best guess is that these young’uns have gotten confused by some Microsoft software allowing both the slash and backslash to be used as directory separators. There was a time when their web browser allowed this and caused some people just learning how to make web sites to make links that were incompatible with anything other than Microsoft software. It seems we have gotten past that bit of confusion from the late 1990’s, yet some software developers get confused when I tell them to use a slash, even though they ought to know them apart to make their division and escape sequences work.

I suppose I might be partly to blame. I regularly use slashes in paths on code that runs on Windows systems because I know that the Windows libraries, like Win32, will interpret the slash as a directory separator. It was a fine decision on Microsoft’s part to assist in porting UNIX software to Windows, and I do use some software at work that started on UNIX and was later ported to Windows. The whole development team does, even if they don’t realize it. So, I use slashes in the paths because they work, and I don’t have to type two characters; using the backslash would start an escape sequence and require another backslash to be interpreted as a single backslash in a string literal.

Still, that doesn’t seem very confusing to me. Software developers need to get this sort of thing straight, so I’m still not clear as to why software developers don’t know what I mean when I say slash. I can understand if they ask about it in the context of a path since not all software on Windows will go both ways, but I get asked outside that context, too. I think when I’m asked which slash to use in the future, I may start answering with use the character for division, or use the character for an escape sequence. If a software developer still doesn’t know what I mean, then they shouldn’t be employed as a software developer. Fortunately, I think it’ll be a while before I find myself working with such an ignorant developer.


May 31, 2009

I’ve been playing Whack-a-Skeeter, a variant of Whack-a-Mole, at home. Starting last Wednesday and continuing through Friday, I had new windows installed on my house. During and since, my place has been in disarray. I suspect it will continue to be into next weekend because I’m not likely to get anything inspected before this upcoming Wednesday and I need to install blinds. My desk has been striped of its monitors, and I’m wondering if I’ll start up my main computer without them to do some software maintenance.

I also moved my PS3 to my living room and connected it to my old CRT TV so that I could watch a DVD I wrote before all this began. My old DVD player didn’t like the disk part way through. I even played Wipeout HD in SD. It lacks some of the graphical wow factor, but it is perfectly playable. Maybe tomorrow I’ll hook up my big monitor in the living room and watch a movie in HD, or just watch it on on my TV in SD because I can. Either way, it’ll be nicer on a couch than the office-style chair in front of my desk, and it’ll benefit from the fine speakers I have in the living room.

Anyway, while the windows were being installed, my house was very open to the outside. Plenty of sawdust from a table saw the crew used on my porch made it into the home, and I’m still cleaning it up. It also rained, so the crew tracked in mud over the floor. Bother. I was cleaning the floor, at least partially, every day until Saturday.

Dead mosquito

Dead mosquito

A fleet of mosquitoes would follow me around outside, so it’s no surprise an armada found its way in. I decided to stalk them for a while. Thanks to the new windows, it is quieter in my house now, and at its quietest I can hear a mosquito in flight a few feet from my ears. I just don’t have a good sense of the direction to the mosquito without seeing it. Most of them I squash with my hands, a few I bagged with my Dust Buster type tool, and I stomped on one. A few times I’ve gotten two in a row (quick successive successful attempts), and once I got two with one blow. Sometimes its a bloody mess, but I’m not tasting them back. Hunting them is enough for me.

I think I might be getting good at this game. I hope so, anyway. There hasn’t been a shortage of mosquitoes for me to go after yet, and I’ve still got plenty of itchy spots. At least I’ve got more motivation for this game than Whack-a-Mole. I always thought that one was rather dumb.

— Jeff, Mosquito Hunter (not to be confused with an exterminator)

False Steps

The Space Race as it might have been

You Control The Action!

High Frontier

the space colony simulation game

Simple Climate

Straightforwardly explaining climate change, so you can read, react and then get on with your life.