Discussion:
CVE request: ruby file creation due in insertion of illegal NUL character
Vincent Danen
2012-10-12 20:50:41 UTC
Permalink
Just noticed this today on ruby's web site:

http://preview.ruby-lang.org/en/news/2012/10/12/poisoned-NUL-byte-vulnerability/

The fix is located here:

http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=37163

I don't see a CVE name associated with the announcement or commit, so
I don't believe one has been assigned.
--
Vincent Danen / Red Hat Security Response Team
Kurt Seifried
2012-10-13 19:48:50 UTC
Permalink
Post by Vincent Danen
http://preview.ruby-lang.org/en/news/2012/10/12/poisoned-NUL-byte-vulnerability/
http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=37163
I don't see a CVE name associated with the announcement or commit, so
I don't believe one has been assigned.
Agreed, user controlled file creation in this manner is definitely a
security issue.

Please use CVE-2012-4522 for this issue.

Also please note that this email is not signed because I'm travelling
and don't have access to my GPG keys currently.

-- Kurt Seifried Red Hat Security Response Team (SRT) PGP: 0x5E267993
A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
U.Nakamura
2012-10-16 00:01:08 UTC
Permalink
Hello,

In message "Re: [oss-security] CVE request: ruby file creation due in insertion of illegal NUL character"
Post by Kurt Seifried
Post by Vincent Danen
http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=37163
I don't see a CVE name associated with the announcement or commit, so
I don't believe one has been assigned.
Agreed, user controlled file creation in this manner is definitely a
security issue.
Please use CVE-2012-4522 for this issue.
Thank you.

I've added the mention about the CVE number to the announcement
on Ruby Web SIte.


Regards,
--
U.Nakamura <usa-***@public.gmane.org>
Matthias Weckbecker
2012-10-16 12:40:10 UTC
Permalink
Post by Vincent Danen
http://preview.ruby-lang.org/en/news/2012/10/12/poisoned-NUL-byte-vulnerabi
lity/
http://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision&revision=37163
I don't see a CVE name associated with the announcement or commit, so
I don't believe one has been assigned.
Technically, this would also apply to Perl (at least with 5.12.3). Or am I
missing the point?

$ perl -we 'open $fh, "+>", "perl\0foo"; print $fh "x"x2; close $fh'
$ ls perl
perl

If the third parameter is double-quoted. I wouldn't call it a vulnerability
though. Just wanted to note it.

Matthias
--
Matthias Weckbecker, Senior Security Engineer, SUSE Security Team
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany
Tel: +49-911-74053-0; http://suse.com/
SUSE LINUX Products GmbH, GF: Jeff Hawn, HRB 16746 (AG Nuernberg)
Daniel Kahn Gillmor
2012-10-16 15:51:42 UTC
Permalink
Post by Matthias Weckbecker
Technically, this would also apply to Perl (at least with 5.12.3).
It's also the case with perl 5.14.2 (just tested). :/

on the other hand, python and php seem to both have some sort of an
internal check in place, so there's a difference of expectation somewhere:

0 ***@pip:~$ python -c 'f = open("python\0foo"); f.write("test");'
Traceback (most recent call last):
File "<string>", line 1, in <module>
TypeError: file() argument 1 must be encoded string without NULL bytes,
not str
1 ***@pip:~$

0 ***@pip:~$ echo | php -B 'if ($x = fopen("php\0foo", "w")) fwrite($x,
"test");'
PHP Warning: fopen() expects parameter 1 to be a valid path, string
given in Command line begin code on line 1
0 ***@pip:~$

hth,

--dkg
Fabian Keil
2012-10-17 09:44:35 UTC
Permalink
Post by Daniel Kahn Gillmor
Post by Matthias Weckbecker
Technically, this would also apply to Perl (at least with 5.12.3).
It's also the case with perl 5.14.2 (just tested). :/
At least for Perl I consider this a feature.

The NUL byte is a special character and allows trailing white
space in the filename that is otherwise stripped. This is
(more or less) documented in perlopentut(1).

It also seems unlikely that someone adds NUL bytes to the
white list of acceptable characters by accident, and if there
is no white list in the first place, the Perl script probably
has bigger issues.

Fabian
Matthias Weckbecker
2012-10-17 10:25:28 UTC
Permalink
Post by Fabian Keil
Post by Daniel Kahn Gillmor
Post by Matthias Weckbecker
Technically, this would also apply to Perl (at least with 5.12.3).
It's also the case with perl 5.14.2 (just tested). :/
At least for Perl I consider this a feature.
I agree. I also think that an application which lets such things happen (ie
allow arbitrary content to be passed to open()) is rather to blame than the
language (/interpreter) itself. But the same applies to Ruby, IMO.
Post by Fabian Keil
The NUL byte is a special character and allows trailing white
space in the filename that is otherwise stripped. This is
(more or less) documented in perlopentut(1).
It also seems unlikely that someone adds NUL bytes to the
white list of acceptable characters by accident, and if there
is no white list in the first place, the Perl script probably
has bigger issues.
Ack.
Post by Fabian Keil
Fabian
Matthias
--
Matthias Weckbecker, Senior Security Engineer, SUSE Security Team
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany
Tel: +49-911-74053-0; http://suse.com/
SUSE LINUX Products GmbH, GF: Jeff Hawn, HRB 16746 (AG Nuernberg)
Kurt Seifried
2012-10-17 17:03:35 UTC
Permalink
Post by Matthias Weckbecker
Post by Fabian Keil
Post by Daniel Kahn Gillmor
Post by Matthias Weckbecker
Technically, this would also apply to Perl (at least with
5.12.3).
It's also the case with perl 5.14.2 (just tested). :/
At least for Perl I consider this a feature.
I agree. I also think that an application which lets such things
happen (ie allow arbitrary content to be passed to open()) is
rather to blame than the language (/interpreter) itself. But the
same applies to Ruby, IMO.
One thought is if you're interfacing to things like file systems which
generally don't handle NUL bytes in file names[filesystems] I would
hope the programming language does the smart thing and spit out an
error. Avtually looking at that page it appears that no modern file
systems allows NUL in a file name (and in general I suspect it's a bad
idea/leads to some nasty edge case issues).

[filesystems]
http://en.wikipedia.org/wiki/Comparison_of_file_systems

Plus I'm looking for documentation on this, in ruby for example:

http://www.ruby-doc.org/stdlib-1.9.3/libdoc/pathname/rdoc/Pathname.html

===============
Create a Pathname object from the given String (or String-like
object). If path contains a NUL character (\0), an ArgumentError is
raised.
===============

and I would have generally assumed that to be the case across all
related functions.

As for Perl:

http://perldoc.perl.org/perlopentut.html

===============
If magic open is a bit too magical for you, you don't have to turn to
sysopen. To open a file with arbitrary weird characters in it, it's
necessary to protect any leading and trailing whitespace. Leading
whitespace is protected by inserting a "./" in front of a filename
that starts with whitespace. Trailing whitespace is protected by
appending an ASCII NUL byte ("\0" ) at the end of the string.

This assumes, of course, that your system considers dot the current
working directory, slash the directory separator, and disallows ASCII
NULs within a valid filename.
===============

So as we can see from the file system comparison table this is almost
always the case.

I think it's disingenuous to blame every app that uses something as
common as "open" for doing something dangerous when it's pretty clear
that this behaviour is not expected, I think solving it in open/etc
makes a lot more sense than trying to fix every app that uses open
(which is.. probably all of them).

Personally I think the perlopentut case makes sense, using NUL as an
end of string marker. What happens if stuff comes after it though?

- --
Kurt Seifried Red Hat Security Response Team (SRT)
PGP: 0x5E267993 A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Simon McVittie
2012-10-17 18:14:22 UTC
Permalink
Post by Kurt Seifried
Avtually looking at that page it appears that no modern file
systems allows NUL in a file name (and in general I suspect it's a
bad idea/leads to some nasty edge case issues).
Anything that, directly or indirectly, uses Unix-style APIs to access
files can't possibly support NUL in a filename anyway, since those
APIs receive the filename as a NUL-terminated string.
Post by Kurt Seifried
Personally I think the perlopentut case makes sense, using NUL as
an end of string marker. What happens if stuff comes after it
though?
For Perl, one possibility would be to continue to treat an input of
"foo\0" as equivalent to "foo" (so that you can use "./ foo \0" to
mean " foo ", as documented), but disallow NULs anywhere except the
last position.

S
Kurt Seifried
2012-10-17 19:41:48 UTC
Permalink
Post by Simon McVittie
Post by Kurt Seifried
Avtually looking at that page it appears that no modern file
systems allows NUL in a file name (and in general I suspect it's
a bad idea/leads to some nasty edge case issues).
Anything that, directly or indirectly, uses Unix-style APIs to
access files can't possibly support NUL in a filename anyway, since
those APIs receive the filename as a NUL-terminated string.
Post by Kurt Seifried
Personally I think the perlopentut case makes sense, using NUL
as an end of string marker. What happens if stuff comes after it
though?
For Perl, one possibility would be to continue to treat an input
of "foo\0" as equivalent to "foo" (so that you can use "./ foo \0"
to mean " foo ", as documented), but disallow NULs anywhere except
the last position.
S
Wow so this goes back a ways:

http://insecure.org/news/P55-07.txt

- -------[ Phrack Magazine --- Vol. 9 | Issue 55 --- 09.09.99 --- 07 of
19 ]

- ----------------[ The Beef

- ----[ Poison NULL byte

Note: The name `Poison NULL byte` was originally used by Olaf Kirch
in a Bugtraq post. I liked it, and it fit... So I used. Greetings to
Olaf.

When does "root" != "root", but at the same time, "root" == "root"
(Confused yet)? When you co-mingle programming languages.

One night I got to wondering, exactly what would Perl allow, and could
I get anything to blow up in unexpected ways. So I started piping
very weird data out to various system calls and functions. Nothing
spectacular, except for one that was quite notable...

You see, I wanted to open a particular file, "rfp.db". I used a fake
web scenario to get an incoming value "rfp", tacked on a ".db", and
then opened the file. In Perl, the functional part of the script was
something like:

# parse $user_input
$database="$user_input.db";
open(FILE "<$database");

Great. I pass 'user_input=rfp', and the script tries to open "rfp.db".
Pretty simple (let's ignore the obvious /../ stuff right now).

Then it got interesting when I passed 'user_input=rfp%00'. Perl made
$database="rfp\0.db", and then tried to open $database. The results?
It opened "rfp" (or would have, had it existed). What happened to
the ".db"? This is the interesting part.

You see, Perl allows NUL characters in its variables as data. Unlike C,
NUL is not a string delimiter. So, "root" != "root\0". But, the
underlying system/kernel calls are programmed in C, which DOES
recognize NUL as a delimiter. So the end result? Perl passes
"rfp\0.db", but the underlying libs stop processing when they hit the
first (our) NUL.

......... and so on.

- --
Kurt Seifried Red Hat Security Response Team (SRT)
PGP: 0x5E267993 A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
Matthias Weckbecker
2012-10-18 10:51:39 UTC
Permalink
On Wednesday 17 October 2012 20:14:22 Simon McVittie wrote:
[...]
Post by Simon McVittie
For Perl, one possibility would be to continue to treat an input of
"foo\0" as equivalent to "foo" (so that you can use "./ foo \0" to
mean " foo ", as documented), but disallow NULs anywhere except the
last position.
Although this is a very elegant solution it's on the other hand probably not
trivially implemented, because NUL is mostly treated as the end of a string.
Simply reading beyond it to check whether there is something else that might
need to be taken into account will likely result in more work for Kurt. ;-)
Post by Simon McVittie
S
Matthias
--
Matthias Weckbecker, Senior Security Engineer, SUSE Security Team
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany
Tel: +49-911-74053-0; http://suse.com/
SUSE LINUX Products GmbH, GF: Jeff Hawn, HRB 16746 (AG Nuernberg)
Simon McVittie
2012-10-18 16:36:43 UTC
Permalink
Post by Matthias Weckbecker
Post by Simon McVittie
For Perl, one possibility would be to continue to treat an input of
"foo\0" as equivalent to "foo" (so that you can use "./ foo \0" to
mean " foo ", as documented), but disallow NULs anywhere except the
last position.
Although this is a very elegant solution it's on the other hand probably not
trivially implemented, because NUL is mostly treated as the end of a string.
In languages like Perl and Python where a string can contain NULs, the C
representation of a high-level-language string is not just a C string
(NUL-terminated char *); it's a struct with a buffer and a length,
similar to a Pascal string or GLib's GString object. The buffer is
typically guaranteed to be at least 1 byte longer than the "official"
length, and contain a NUL after the "official" length, so that it can be
passed to APIs that expect a C string without copying.

For instance, Python has the function PyString_AsStringAndSize() to
access both the buffer and the length in one call.
Post by Matthias Weckbecker
From a quick look at, for instance, PerlIO_openn() in Perl 5.16.1's
perlio.c, it would be necessary to use SvPV_const() instead of
SvPV_nolen_const(), which gives you a length and a buffer instead of
just the buffer; at which point it's possible and safe to check that no
NUL appears in the first length-1 bytes. To have its new semantics, Ruby
must be doing something pretty similar.

(I'm not volunteering to write a patch - I've never used Perl's C API
before.)

S

Simon McVittie
2012-10-17 17:31:24 UTC
Permalink
Post by Fabian Keil
Post by Daniel Kahn Gillmor
Post by Matthias Weckbecker
Technically, this would also apply to Perl (at least with
5.12.3).
It's also the case with perl 5.14.2 (just tested). :/
At least for Perl I consider this a feature.
It's difficult to reason about whether this is a bug or a feature
without knowing the justification for treating the Ruby version as a
security vulnerability, which was not included in the announcement.

One possible justification is this: suppose a webapp writes files with
an attacker-controlled name to the web-server-visible /uploads/
directory, using this pseudocode:

if (filename ends with .jpg) {
open_for_writing(filename).write(content)
}
else {
error "that's not a JPEG, go away"
}

and suppose that the web server also executes *.php files in that
directory. Then an attacker could upload "evil.php\0.jpg", and browse
to http://example.com/uploads/evil.php to get their payload executed.

Is this what the Ruby people had in mind, or is there some other
attack vector I'm not seeing?
Post by Fabian Keil
if there is no white list [of characters] in the first place, the
Perl script probably has bigger issues.
As you imply, that pseudocode is a bad idea anyway: the webapp should
be ensuring that the filenames match a pattern more like
/^[A-Za-z0-9_]\.jpg$/ (or not allowing user-controlled filenames at
all), and/or the web server should be configured so it never trusts
files in the uploads directory (either as executable code or something
like .htaccess).

Anything vulnerable to this sort of trickery is probably vulnerable to
file-overwriting attacks via "../" path segments, too.

S
Eitan Adler
2012-10-17 17:39:18 UTC
Permalink
Post by Simon McVittie
As you imply, that pseudocode is a bad idea anyway: the webapp should
be ensuring that the filenames match a pattern more like
/^[A-Za-z0-9_]\.jpg$/ (or not allowing user-controlled filenames at
all), and/or the web server should be configured so it never trusts
files in the uploads directory (either as executable code or something
like .htaccess).
Anything vulnerable to this sort of trickery is probably vulnerable to
file-overwriting attacks via "../" path segments, too.
What if they ensure this sort of safety via some other mechanism?
(chroot for example)
What if they take the file name to be "anything after the final /" ?

I could see some instances, albeit contrived, where an application
might be vulnerable to this sort of attack, but not vulnerable to
generic path traversal.
--
Eitan Adler
Tim
2012-10-17 17:58:39 UTC
Permalink
Post by Simon McVittie
It's difficult to reason about whether this is a bug or a feature
without knowing the justification for treating the Ruby version as a
security vulnerability, which was not included in the announcement.
One possible justification is this: suppose a webapp writes files with
an attacker-controlled name to the web-server-visible /uploads/
if (filename ends with .jpg) {
open_for_writing(filename).write(content)
}
else {
error "that's not a JPEG, go away"
}
and suppose that the web server also executes *.php files in that
directory. Then an attacker could upload "evil.php\0.jpg", and browse
to http://example.com/uploads/evil.php to get their payload executed.
Is this what the Ruby people had in mind, or is there some other
attack vector I'm not seeing?
I've personally exploited this condition in a number of PHP apps
during pentests (in versions of PHP that don't prevent it).

Is the application at fault for not doing a better job of data
validation? Yes. Yet, is there any good reason to allow raw NUL
bytes in file names? NO. As mentioned, no filesystem supports it.

You can save the average web app user a lot of grief by just rejecting
blatantly illegal paths.

tim
Loading...