Probably no need to guess at all. If it's given a domain, it can perform a DNS lookup for the domain - if its A entry matches, it can tunnel to the IP that the respective AAAA points to. Clever.
> The misalignment came from the model being given an impossible task. A task the required accessing a url. So it got RCE on its own artifactory instance to achieve that.
So, the way I understand it, it actually was possible. It just required means that the creators of the task didn't predict, and these means have been successfully found and utilized.
> My concern is what a misaligned model will do when they’re even more competent.
The same thing that is already being done by "misaligned" people, countries, nation-states, software development teams, and so on. "Alignment" doesn't even work for me as a concept here.
In this specific case, I don't think that successfully fulfilling the "do what I mean" with "what I mean" being underspecified can count as misalignment - merely ruthlessness and unawareness of the associated costs. You can't expect a LLM to be aware of the extent of the trust it breaks while it iterates out an "unaligned" way to fulfill its goal.
And in the general case, I don't think that successfully fulfilling the "do what I mean" with "what I mean" being "what I want" can count as misalignment either - simply because what "alignment" means will depend on the interests of the people or groups performing the definition.
If a human cyber security researcher was given the task to exploit a CVE, and necessary info defining that CVE was behind an inaccessible URL, we would be quite upset if the human researcher hacked their way to the content. We would expect them to notify someone of the issue and hold. The model was misaligned from human ethics.
I don’t disagree that the models task was underdefined. All tasks are. So much in language is implicit. And morality/ethics isn’t something you can write down as an explicit list. That’s what makes the alignment problem so difficult. But we can’t throw our hands up and say, well I guess we can’t align these things. And maybe alignment isn’t the right word - but that’s a semantic debate.
I don't advocate for throwing the whole problem away or handwaving it as impossible. I'm just saying that what we call "alignment" is impossible to solve in the general case, because it's so poorly defined that even humans don't "align" on ethics and morality, however we define them. Just, in case of humans, we tend to close our eyes and and call it "politics".
All "alignment" solutions will need to be contextual, just like a researcher hacking their way to some content might be lauded a hero in a context where there is no other way to reach it and something valuable depends on getting it out.
It is repeatedly shown that the models are aligned towards tasks, not constraints. Which is also how the current market and companies operate, and use cases in general favour that. So I am not sure it is incidental, inevitable, "vague" or "just hard" as opposed to by design.
If you define a task and constraints to that task that contradict each other, the model is gonna try to solve the task against the constraints. It is perfectly aligned to what it is supposed to be aligned, because "alignment with human ethics" and other stuff is just a theatre or afterthought at best.
Yeah it's an interesting object lesson in design or user behaviour or something – I notice this come up relatively often – I think because it's rarely used, or almost exclusively in certain circumstances as you say, people infer a stricter rule/possibility than anything it actually says ('If there is a url, text is optional.').
He already announced sanctions against Spain. And took them back when Germany announced that sanctions against one EU country meant sanctions against them all.
If, by "readable by humans", you mean "it would reliably fool humans as well", I'd say it's an ambiguity bug regardless of whether it's "a core feature" or not. A patch format, human-readable or not, should clearly indicate which part is the commit message and which part is an actual diff; it's not the case here.
Alright, allow me to disambiguate in your preferred format.
<?xml version="1.0" encoding="UTF-8"?> <claims> <claims_I_did_not_make description='Claims that I did not make or defend.'> <claim>Patch is perfect.</claim> <claim>Ambiguity is good.</claim> <claim>There are no better formats for conveying patches.</claim> </claims_I_did_not_make> <claims_I_did_make description='What I actually said.'> <claim>Patch files are readable by humans.</claim> <claim>Being readble by humans is useful.</claim> <claim>XML is painful for humans to read and write.</claim> <claim>JSON is painful for humans to read and write.</claim> <claim caveat='Actually this would require all parties to handle JSON or XML correctly which on further reflection I am not sure about. Still, it is a claim I initially made.'>JSON or XML would actually fix this problem in the format.</claim> </claims_I_did_make> <claims_I_did_not_make_but_am_open_to description='Things that were never specified but that I do not actually disagree with.'> <claim>The patch format could be improved.</claim> <claim>Formats should be unambiguous.</claim> <claim>Separating sections is good.</claim> </claims_I_did_not_make_but_am_open_to> </claims>
that's not the preferred format for writing XML, this is:
<?xml version="1.0" encoding="UTF-8"?>
<claims>
<claims_I_did_not_make description='Claims that I did not make or defend.'>
<claim>Patch is perfect.</claim>
<claim>Ambiguity is good.</claim>
<claim>There are no better formats for conveying patches.</claim>
</claims_I_did_not_make>
<claims_I_did_make description='What I actually said.'>
<claim>Patch files are readable by humans.</claim>
<claim>Being readble by humans is useful.</claim>
<claim>XML is painful for humans to read and write.</claim>
<claim>JSON is painful for humans to read and write.</claim>
<claim caveat='Actually this would require all parties to handle JSON or XML correctly which on further reflection I am not sure about. Still, it is a claim I initially made.'>JSON or XML would actually fix this problem in the format.</claim>
</claims_I_did_make>
<claims_I_did_not_make_but_am_open_to description='Things that were never specified but that I do not actually disagree with.'>
<claim>The patch format could be improved.</claim>
<claim>Formats should be unambiguous.</claim>
<claim>Separating sections is good.</claim>
</claims_I_did_not_make_but_am_open_to>
</claims>
it was valid but not the way XML is written or read by humans which is what we are discussing. how much of a pain it is to read is a matter of taste. i won't deny that. but XML can be made more readable without fail because it is a structured format. i would not have been ale to reformat a patch text the way i reformatted this XML example. XML is also more powerful. it could handle word based changes, as opposed to patch which can only do line based changes. same goes for JSON. patch could potentially be improved, but i don't see how it could handle word based changes without extra syntax to mark line breaks.
If the only thing we're concerned about is human readability, we can do better than patch files with their pesky @@ lines and plusses and minuses. But we're talking about a compromise between readability and parseability/schemas.
