[oss-security] bounties

[Justin Ferguson]

Hello,

I was curious about peoples experiences with bug bounties particularly
those through the prominent clearing houses for them. My experience is
that I have been either ripped off or extremely slow-walked in payment
that was substantially below the listed payout in every single
instance. I'm curious how accurately that reflects other peoples
experiences.

In the first series of findings, the vendor, a popular open source
component simply patched the bugs and refused to close the tickets
triggering payout for over a year. Attempts at resolving this through
the clearing houses support produced an endless series of excuses
mostly revolving around their not having any insight into their own
database (which is probably true). After a year or so, the ticket was
finally closed and the pay out several hundred dollars less than the
enumerated payout. I refused the bounty citing these complications and
insisted that the finding as a work for hire that was rejected and
requested that the patch be reverted as a result, which was just
ignored.

In the second series, the vendor, a prominent hardware company, stated
that a one line fix with no usability impact (the patch is to move the
line up one line so that it is included in the mutex lock) was found
and "partly fixed" over a month prior and that a full patch should be
released soon. That was several months ago and looking through their
reports, their public repositories, et cetera it appears to be totally
and entirely something they made up as the bug still exists. This
meshes with my thoughts that there even was such a thing as a partial
fix for x() mutex.lock() vs mutex.lock() x();.

In the third instance, the vendor, an anti-virus vendor in Europe,
stated that they were not able to reproduce the issue and didn't see
any issue. There were multiple things reported to them and their
circumstances were different as a context switch meant I was turning
in incomplete work just to attempt to get the issues patched. After
months of them coming back and asking the same question repeatedly,
being told the same answer repeatedly and continually ignoring very
basic questions about their attempts to reproduce, they closed the
matter as not reproducible. Upon further review, they could not have
possibly reviewed anything as the issue is blatantly clear and obvious
implying that they must not have even looked at the matter. In
additional findings reported to them, they've outright ignored the
matter entirely.

Thus, my experience has thus far been that bounties, particularly
those through the clearing houses are basically enabling a 1990s
pre-full-disclosure series of processes under the pretense of the
opposite, but in practice mostly just ripping works for hire off. This
clearly isn't the case across the board, but its been true in every
instance of my participation.

-me

[Solar Designer]

Hi,

Moderator hat on:

I'm sorry for the delay in the moderation decision on this one message.
We overlooked it at first for its spam-like Subject, only noticing what
it actually was when follow-ups started arriving today.

I am unhappy about the cross-post (this is against the published
oss-security guidelines, but it sometimes happens anyway), even though
this is what ultimately enabled the non-spam detection in this case.

I am also unhappy about the lack of focus on Open Source (but not total
lack of relation to Open Source, which is why the message is approved)
in this message and in follow-ups we might receive via full-disclosure.

While the original message looks like something we'd need to approve
despite of the above issues (so I did), I currently have no plans to
approve any follow-ups we might receive via full-disclosure. Currently
in the moderation queue, and not expected to be approved, are a very
brief reply by another person confirming Justin's criticism and giving a
link to the person's Medium blog post (at first look, only tangentially
related to Open Source - pertaining to proprietary products that use
Open Source components), Justin's reply to the reply (a rant and no
longer a question/request for other people's experience; also no mention
of Open Source nor any specific software nor vendor at all), and
Justin's out-of-context copying of a reply (repeatedly mentioning "rip
off") to a who-knows-what vendor about a who-knows-what product (these
things are not immediately clear from the message). Even though not
approved here, there's a chance we'll see those messages on
full-disclosure, depending on that other list's moderators' decisions.

So quite possibly the thread on oss-security will end here, unless
someone will post something of greater relevance and/or higher quality
than what's in those follow-ups I mentioned above.

Post by Justin Ferguson
I was curious about peoples experiences with bug bounties particularly
those through the prominent clearing houses for them. My experience is
that I have been either ripped off or extremely slow-walked in payment
that was substantially below the listed payout in every single
instance. I'm curious how accurately that reflects other peoples
experiences.

I have very little experience, in part because I've never been hunting
for bug bounties. I only recall receiving a bug bounty three times so
far, from three different vendors. In the first case, I didn't know the
vendor had a bug bounty program (which was very uncommon at the time -
1999). In other two cases (in 2010s), I was aware of the bug bounty
programs (and in one case went via HackerOne, as a test of that
platform, which worked perfectly) but the issues I was submitting were
clearly beyond scope, yet I was paid the bounties anyway. The amounts
were moderate, but it was very kind of those vendors to offer anything
at all. So no complaints from me.

I do hear that others have all sorts of different experience. There's
also criticism from many vendors about the behavior of bug reporters.
Once a vendor offers a bug bounty, they commonly receive lots of crappy
reports, accusations, etc. Unfortunately. (Yet by saying this I don't
mean to defend any vendor not honoring their bug bounty terms.)

Alexander

[Jeremy Stanley]

[Full Disclosure ML dropped from followup]

Post by Justin Ferguson
I was curious about peoples experiences with bug bounties
particularly those through the prominent clearing houses for them.
My experience is that I have been either ripped off or extremely
slow-walked in payment that was substantially below the listed
payout in every single instance. I'm curious how accurately that
reflects other peoples experiences.

[...]

As someone handling intake of suspected vulnerability reports for a
large community of free/libre open source software projects, I've
seen another side of it. The projects I work on have been
incorrectly added and re-added to lists of supposed bug bounty
targets over the years, and it's caused us to deal with floods of
useless reports from everyone who can figure out how to run a static
code analyzer, fuzzer or vulnerability scanner (and also people who
can't even figure out the difference between the projects and the
code which powers their community Web sites).

Convincing the people who maintain those clearing house lists to
de-list your projects can be a challenge, as they're just as likely
to ignore you, or even simply be abandoned Web sites with nobody at
the helm. If this is the sort of experience other projects endure, I
can't imagine why any would willingly put themselves on such bounty
registries. I have much more interest in dealing with reports of
suspected vulnerabilities from engaged users of the software than
from people out to make a quick buck, disinterested in even
following up enough on the bugs they think they've found to
determine they're unreachable cruft or even intentional features of
the software.

[Justin Ferguson]

note to moderator: the last paragraph is likely what you'd like to
read first, then the remainder of the exchange

Post by Jeremy Stanley
As someone handling intake of suspected vulnerability reports for a
large community of free/libre open source software projects, I've
seen another side of it. The projects I work on have been
incorrectly added and re-added to lists of supposed bug bounty
targets over the years, and it's caused us to deal with floods of
useless reports from everyone who can figure out how to run a static
code analyzer, fuzzer or vulnerability scanner (and also people who
can't even figure out the difference between the projects and the
code which powers their community Web sites).

Well this is a different situation overall, but what I would say is
that when you engage in one of the clearing houses that you are
engaging in a business transaction. A lot of companies do this so that
they don't have to pay for a full local security team and bill it as a
cheaper alternative. That they then have to deal with a lot of cruft
doesn't really alleviate their responsibilities-- so the vendor wants
their cake and to eat it to essentially.

In all of my years, I've actually only had a single entity do really
what they were supposed to do when receiving a submission: OpenBSD.
While looking for something else I ran across a vulnerability in their
IKEd implementation that was an issue with a return value checked
incorrectly on a cryptographic hash verification. I wasn't soliciting
money or even a CVE, I was just reporting an issue to them. Under such
conditions, most vendors seem to want to brow beat you into doing
their work for them despite the obviousness of the issue. The OpenBSD
team just said "oh you're right" and patched their issue.

Post by Jeremy Stanley
I have much more interest in dealing with reports of
suspected vulnerabilities from engaged users of the software than
from people out to make a quick buck, disinterested in even
following up enough on the bugs they think they've found to
determine they're unreachable cruft or even intentional features of
the software.

Well your circumstances are different, and this is where Alexander was
entirely spot on in his commentary about OSS. One of the things
removed outlined that I've had some fair amount of issues in
traditional employment as a byproduct of what's been termed as a
"distributed denial of service" attack (there's a lot of political
context that I am leaving out, I've never participated in a DDoS or
anything of the sort but as we all know from our own industry there's
a lot of budget in security and so sometimes things don't necessarily
have to have much to do with a particular/person).

As a result, I've dabbled in augmenting my income through bounties--
it seems reasonable, it's something I should be able to do, I'm not
submitting ASAN output or running AFL and handing a bunch of cruft
over. In the most recent instance, when I task switched to something
more pressing, I handed over data from things I wasn't even looking
for but noticed while I was trying to find the parsers in a myriad of
DLLs and similar as a courtesy to the vendor and their customers. For
instance, while trying to discern an aspect about the usage of
UNICODE_STRING structures as part of trying to discern whether there
was a basic buffer overflow in command line processing accessible only
via a pipe that exchanges JSON (privsep essentially), I triggered a
null pointer dereference in the kernel. I briefly looked at it,
discerned that it was a null pointer dereference and put it away while
continuing to sort through the binaries towards what I was attempting
to focus on.

When I changed tasks, I submitted a couple IDA screenshots, the
program that at the time was triggering a BSoD consistently, a brief
explanation of each, excerpts of the crash dump and explained that I
didn't necessarily expect to be paid money and was submitting partial
work as a courtesy to their customers.

Over the next two months, I would intermittently receive "are you
going to do my job for me?" type responses while the vendor ignored my
basic questions-- "does the PoC not trigger a BSoD for you?"
continually solicited commentary that was carefully worded and avoided
answering the question "We don't see any reason why this section of
code would have that problem" essentially.

Eventually I suggested after being poked repeatedly on the matter,
something I clearly had moved on from and repeatedly stated as such,
that moving forward I should probably just post anything I find of
theirs on FD, which not only solicited commentary about my being
'unethical', but then follow-ups about my ethics from the clearing
house. For the record, this clearing house maintains a director of
security that I used to be employed with at a company that once tried
to sell an OpenSSL bug to the CIA and wanted to do so without any sort
of customer notification-- they equally put into my contract after
telling me that they were a small company that couldn't afford a
pay-rate over $70k a year (they absolutely can) that I needed to find
and write and sell to the United States Government 12 exploits a year,
in order to qualify for the pay rate that they were giving everyone
else-- so I find being told I am unethical a little annoying.

After this point, I reviewed the finding that they couldn't find a
reason why the null pointer dereference would occur and found pretty
quickly essentially the following code construct:

bool
function afunction(T** ptr) {
if (ptr)
if (!
doComplexActionThatIsHighlyDependentUponOverallSystemState(ptr))
return false;
if (! *ptr)
return true;
if (!
doComplexActionThatIsHighlyDependentUponOverallSystemState(*ptr))
return false;
...
if (! afunction(ptr))
return false;
if (*ptr == 0x22) <- null pointer dereference here

Thus, they pretty clearly didn't even look at their code, and because
of EULAs and ToS, I cannot really discuss that I've had this and
similar issues in almost/100% of bounty submissions, so they're sort
of just acting as inhibitors to security sometimes.

Because of my industry experience, I have a hinting suspicion, and I
don't think this finding is an example of it, but I have a hinting
suspicion that they're being used to some extent by my governments
intelligence community-- when I worked as part of a JTF that included
NSA personnel, Microsoft was providing them with a "beta patch" that
provided them verified exploitable bugs reported to them with
reproduction instructions under the pretext of beta testing patches
before release-- "-Xday" if you will (as opposed to 0day). As noted
above, I've had other security vendors explicitly sell or attempt to
sell vulnerabilities to our intelligence services, an IPS vendor I
previously worked for had some amount of issues reported to vendors
that would sort of just vanish and I specifically enumerated a
particular Active Directory report.

I noticed another entity, OSTIF (https://ostif.org) previously claimed
that they were having issues with Google Mail servers that appeared to
be server side, I noticed it because I've experienced similar myself.
Moreover, I had problems when researching breaking of Google
recaptcha's that twice over an extensive period of time that when I
started working on breaking the that Google then made minor tweaks and
variations that broke or impacted the research. In review, other
researchers have had similar experiences and while there is nothing
conclusive, I was at the time conversing with a private party over
Google email about the subject.

Thus I am asking in a larger context about other peoples experiences
with such things, and while I would agree that this subject matter is
not particularly on-topic for the list, I would think that there
effectively are not unmoderated lists, no place to really ask this
question in a substantial manner despite the importance of it, and of
crowd-sourcing other peoples experiences would be concerning to an
industry as a whole.

Post by Jeremy Stanley
[Full Disclosure ML dropped from followup]

Post by Justin Ferguson
I was curious about peoples experiences with bug bounties
particularly those through the prominent clearing houses for them.
My experience is that I have been either ripped off or extremely
slow-walked in payment that was substantially below the listed
payout in every single instance. I'm curious how accurately that
reflects other peoples experiences.

[...]
As someone handling intake of suspected vulnerability reports for a
large community of free/libre open source software projects, I've
seen another side of it. The projects I work on have been
incorrectly added and re-added to lists of supposed bug bounty
targets over the years, and it's caused us to deal with floods of
useless reports from everyone who can figure out how to run a static
code analyzer, fuzzer or vulnerability scanner (and also people who
can't even figure out the difference between the projects and the
code which powers their community Web sites).
Convincing the people who maintain those clearing house lists to
de-list your projects can be a challenge, as they're just as likely
to ignore you, or even simply be abandoned Web sites with nobody at
the helm. If this is the sort of experience other projects endure, I
can't imagine why any would willingly put themselves on such bounty
registries. I have much more interest in dealing with reports of
suspected vulnerabilities from engaged users of the software than
from people out to make a quick buck, disinterested in even
following up enough on the bugs they think they've found to
determine they're unreachable cruft or even intentional features of
the software.
--
Jeremy Stanley

[Solar Designer]

Post by Justin Ferguson
Thus I am asking in a larger context about other peoples experiences
with such things,

OK. You already did that, and people are free to share their experience
(if on-topic for this list). You also shared your experience (which
happens to be only partially on-topic for this list). As a moderator, I
think that's enough contribution from you to this thread.

Post by Justin Ferguson
and while I would agree that this subject matter is
not particularly on-topic for the list, I would think that there
effectively are not unmoderated lists, no place to really ask this
question in a substantial manner despite the importance of it, and of
crowd-sourcing other peoples experiences would be concerning to an
industry as a whole.

While I also regret that there's no longer an unmoderated
full-disclosure list, I understand why there is not. Besides needing to
filter out all the automated spam (which already makes the list
technically moderated), all sorts of semi-on-topic crap ends up being
posted to truly unmoderated lists like that - personal attacks, doxing
(and when you ask for removal of that content from archives later, you
sort of engage in moderation too), conspiracy theories and dubious facts
(and good luck figuring out what's real and what's not), etc. That
said, you may host a new list like that, and I'd be happy to be able to
redirect discussions that are unsuitable in here to there.

Alexander