"...the kind of analysis the program is able to do is past the point where technology looks like magic. I don’t know how you get here from “predict the next word.”"
The best analogy I can think of is the transistor. I still don't understand how a bunch of, essentially, on-and-off switches can result in, for example, a movie streamed to my living room tv. But somehow it does. Still, definitely indistinguishable from magic. (And these are just the early days – the "Eniac" of LLM days...)
Oh, pshaw! It's not magic. It's the theory of dark in action. Light bulbs are really dark suckers which send the dark through the wires to the power plant where the dark is shot skyward from those big chimneys, and eventually the sun sucks up all the dark and has to take a break to digest it. The proof is how much dark is emitted when you short out a circuit in anything electrical.
Thanks for pointing this out. Hard to keep on top of tools. Was "meh" for me on a shortish paper I just submitted. Maybe a bit worse than Claude Opus 4.6 in project mode with all my research in the working directory. Perhaps on par with ChatGPT cold.
It is very verbose, which to us classists indicates inability to condense! That in turn indicates inability to precisely match language to ideas. Thus, taking 100 words to try to capture your meaning because you cannot do it in 10.
Good timing. I just tried the "free" version on a very technical finance paper that I am in the process of refereeing. I fed it the .docx that had been provided by the editor and ran it through their free (abridged) trial. Unfortunately, it appears that the authors cut and pasted equations from another (unknown) program into Word and, consequently, Refine saw them as pictures and ignored them. So the comments from Refine largely centered on the paper's text, Nonetheless, it didn't seem to fully grasp the primary thrust of the paper from the text. So, Refine's feedback was of marginal help. It pointed out a weak spots associated with estimation techniques that were mentioned in the body of the paper, but it probably only saved a few minutes of review time. Not a fair test of the program, but perhaps a cautionary tale. I think the only way to reliably test the program is to test it on early version of one's own research so that inputs can be controlled.
"...the kind of analysis the program is able to do is past the point where technology looks like magic. I don’t know how you get here from “predict the next word.”"
The best analogy I can think of is the transistor. I still don't understand how a bunch of, essentially, on-and-off switches can result in, for example, a movie streamed to my living room tv. But somehow it does. Still, definitely indistinguishable from magic. (And these are just the early days – the "Eniac" of LLM days...)
It is even more perplexing how we got to where we are from "pass on your genes."
You're reminding me of the classic Calvin and Hobbes strip, John and Mike!
https://www.reddit.com/r/calvinandhobbes/comments/14y2kvr/dad_coming_through_with_the_explanation/#lightbox
Oh, pshaw! It's not magic. It's the theory of dark in action. Light bulbs are really dark suckers which send the dark through the wires to the power plant where the dark is shot skyward from those big chimneys, and eventually the sun sucks up all the dark and has to take a break to digest it. The proof is how much dark is emitted when you short out a circuit in anything electrical.
Thanks for pointing this out. Hard to keep on top of tools. Was "meh" for me on a shortish paper I just submitted. Maybe a bit worse than Claude Opus 4.6 in project mode with all my research in the working directory. Perhaps on par with ChatGPT cold.
Just out of curiosity, what type of paper was it/what area? We'd happily give you some free reviews to see how you feel about it on a longer paper.
it does shine most on a project like John's 80-pager.
Would be happy to chat - feel free to ping me at ben@refine.ink
IMO, the human still has value. Refine goes for narrower critiques, but sometimes you need to disagree with large choices.
I say “still has value”, but of course, statement only valid for the next two years!
That AI tool left a really good comment about your critique of monetarism, and you didn’t listen to it!
It is very verbose, which to us classists indicates inability to condense! That in turn indicates inability to precisely match language to ideas. Thus, taking 100 words to try to capture your meaning because you cannot do it in 10.
Good timing. I just tried the "free" version on a very technical finance paper that I am in the process of refereeing. I fed it the .docx that had been provided by the editor and ran it through their free (abridged) trial. Unfortunately, it appears that the authors cut and pasted equations from another (unknown) program into Word and, consequently, Refine saw them as pictures and ignored them. So the comments from Refine largely centered on the paper's text, Nonetheless, it didn't seem to fully grasp the primary thrust of the paper from the text. So, Refine's feedback was of marginal help. It pointed out a weak spots associated with estimation techniques that were mentioned in the body of the paper, but it probably only saved a few minutes of review time. Not a fair test of the program, but perhaps a cautionary tale. I think the only way to reliably test the program is to test it on early version of one's own research so that inputs can be controlled.