Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs

Researchers introduced a method called cross-model in-context neurofeedback to study activation control in large language models (LLMs), finding that such control is limited. This approach sheds light on the internal dynamics of LLMs but highlights constraints in manipulating their activations.