LnRiLWZpZWxke21hcmdpbi1ib3R0b206MC43NmVtfS50Yi1maWVsZC0tbGVmdHt0ZXh0LWFsaWduOmxlZnR9LnRiLWZpZWxkLS1jZW50ZXJ7dGV4dC1hbGlnbjpjZW50ZXJ9LnRiLWZpZWxkLS1yaWdodHt0ZXh0LWFsaWduOnJpZ2h0fS50Yi1maWVsZF9fc2t5cGVfcHJldmlld3twYWRkaW5nOjEwcHggMjBweDtib3JkZXItcmFkaXVzOjNweDtjb2xvcjojZmZmO2JhY2tncm91bmQ6IzAwYWZlZTtkaXNwbGF5OmlubGluZS1ibG9ja311bC5nbGlkZV9fc2xpZGVze21hcmdpbjowfQ==
LnRiLWNvbnRhaW5lciAudGItY29udGFpbmVyLWlubmVye3dpZHRoOjEwMCU7bWFyZ2luOjAgYXV0b30gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWNvbnRhaW5lci50Yi1jb250YWluZXJbZGF0YS10b29sc2V0LWJsb2Nrcy1jb250YWluZXI9IjhjMzU0YWFmNDlmYTlkYjU1ZDJhZWZiNzM3OGRiNzFiIl0geyBwYWRkaW5nOiAyNXB4O21hcmdpbi1sZWZ0OiAxNSU7IH0gLnRiLWZpZWxkW2RhdGEtdG9vbHNldC1ibG9ja3MtZmllbGQ9IjRhNmE5ZTQwMzE2OWY3NDEzMjc4N2QzZmI2NDM3MzJkIl0geyBwYWRkaW5nLXRvcDogMHB4O21hcmdpbi10b3A6IC0yNXB4O21hcmdpbi1sZWZ0OiAxMCU7IH0gLnRiLWZpZWxkW2RhdGEtdG9vbHNldC1ibG9ja3MtZmllbGQ9IjRhNmE5ZTQwMzE2OWY3NDEzMjc4N2QzZmI2NDM3MzJkIl0gYSB7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgfSAgLnRiLWNvbnRhaW5lciAudGItY29udGFpbmVyLWlubmVye3dpZHRoOjEwMCU7bWFyZ2luOjAgYXV0b30gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWNvbnRhaW5lci50Yi1jb250YWluZXJbZGF0YS10b29sc2V0LWJsb2Nrcy1jb250YWluZXI9IjFjY2Y0NDZmODE0ZGE0MGI1MTM0MTQ2ZDE0YmQ1NDQxIl0geyBwYWRkaW5nOiAyNXB4IDEwcHggMjVweCAxMHB4O21hcmdpbi1yaWdodDogMTAlO21hcmdpbi1sZWZ0OiAxMCU7IH0gLnRiLWdyaWQsLnRiLWdyaWQ+LmJsb2NrLWVkaXRvci1pbm5lci1ibG9ja3M+LmJsb2NrLWVkaXRvci1ibG9jay1saXN0X19sYXlvdXR7ZGlzcGxheTpncmlkO2dyaWQtcm93LWdhcDoyNXB4O2dyaWQtY29sdW1uLWdhcDoyNXB4fS50Yi1ncmlkLWl0ZW17YmFja2dyb3VuZDojZDM4YTAzO3BhZGRpbmc6MzBweH0udGItZ3JpZC1jb2x1bW57ZmxleC13cmFwOndyYXB9LnRiLWdyaWQtY29sdW1uPip7d2lkdGg6MTAwJX0udGItZ3JpZC1jb2x1bW4udGItZ3JpZC1hbGlnbi10b3B7d2lkdGg6MTAwJTtkaXNwbGF5OmZsZXg7YWxpZ24tY29udGVudDpmbGV4LXN0YXJ0fS50Yi1ncmlkLWNvbHVtbi50Yi1ncmlkLWFsaWduLWNlbnRlcnt3aWR0aDoxMDAlO2Rpc3BsYXk6ZmxleDthbGlnbi1jb250ZW50OmNlbnRlcn0udGItZ3JpZC1jb2x1bW4udGItZ3JpZC1hbGlnbi1ib3R0b217d2lkdGg6MTAwJTtkaXNwbGF5OmZsZXg7YWxpZ24tY29udGVudDpmbGV4LWVuZH0gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWdyaWQudGItZ3JpZFtkYXRhLXRvb2xzZXQtYmxvY2tzLWdyaWQ9IjgzMWE4MzFkNWUzY2E2MmMyYTFjNjI1ZDg4NjM1ODgzIl0geyBncmlkLXRlbXBsYXRlLWNvbHVtbnM6IG1pbm1heCgwLCAwLjNmcikgbWlubWF4KDAsIDAuN2ZyKTtncmlkLWF1dG8tZmxvdzogcm93IH0gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWdyaWQudGItZ3JpZFtkYXRhLXRvb2xzZXQtYmxvY2tzLWdyaWQ9IjgzMWE4MzFkNWUzY2E2MmMyYTFjNjI1ZDg4NjM1ODgzIl0gPiAudGItZ3JpZC1jb2x1bW46bnRoLW9mLXR5cGUoMm4gKyAxKSB7IGdyaWQtY29sdW1uOiAxIH0gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWdyaWQudGItZ3JpZFtkYXRhLXRvb2xzZXQtYmxvY2tzLWdyaWQ9IjgzMWE4MzFkNWUzY2E2MmMyYTFjNjI1ZDg4NjM1ODgzIl0gPiAudGItZ3JpZC1jb2x1bW46bnRoLW9mLXR5cGUoMm4gKyAyKSB7IGdyaWQtY29sdW1uOiAyIH0gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWdyaWQtY29sdW1uLnRiLWdyaWQtY29sdW1uW2RhdGEtdG9vbHNldC1ibG9ja3MtZ3JpZC1jb2x1bW49IjMwMzRmYmU4ODZjMTEwNTRlOTViNDZiMDlkM2U0MTEyIl0geyBkaXNwbGF5OiBmbGV4OyB9IC50Yi1pbWFnZXtwb3NpdGlvbjpyZWxhdGl2ZTt0cmFuc2l0aW9uOnRyYW5zZm9ybSAwLjI1cyBlYXNlfS53cC1ibG9jay1pbWFnZSAudGItaW1hZ2UuYWxpZ25jZW50ZXJ7bWFyZ2luLWxlZnQ6YXV0bzttYXJnaW4tcmlnaHQ6YXV0b30udGItaW1hZ2UgaW1ne21heC13aWR0aDoxMDAlO2hlaWdodDphdXRvO3dpZHRoOmF1dG87dHJhbnNpdGlvbjp0cmFuc2Zvcm0gMC4yNXMgZWFzZX0udGItaW1hZ2UgLnRiLWltYWdlLWNhcHRpb24tZml0LXRvLWltYWdle2Rpc3BsYXk6dGFibGV9LnRiLWltYWdlIC50Yi1pbWFnZS1jYXB0aW9uLWZpdC10by1pbWFnZSAudGItaW1hZ2UtY2FwdGlvbntkaXNwbGF5OnRhYmxlLWNhcHRpb247Y2FwdGlvbi1zaWRlOmJvdHRvbX0gLndwLWJsb2NrLWltYWdlLnRiLWltYWdlW2RhdGEtdG9vbHNldC1ibG9ja3MtaW1hZ2U9IjU5MWIwYWJkOTZiOGNhMDdiMzk4N2I5OTlkNzBhZTYxIl0geyBtYXgtd2lkdGg6IDEwMCU7IH0gLnRiLWNvbnRhaW5lciAudGItY29udGFpbmVyLWlubmVye3dpZHRoOjEwMCU7bWFyZ2luOjAgYXV0b30gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWNvbnRhaW5lci50Yi1jb250YWluZXJbZGF0YS10b29sc2V0LWJsb2Nrcy1jb250YWluZXI9ImQ0YWI4NzFmMTlkMThlNDdlN2JkNzQzMmMxZDQ4YzU5Il0geyBwYWRkaW5nOiAwcHg7bWFyZ2luOiAyNXB4IDUwcHggMHB4IDUwcHg7IH0gLnRiLWZpZWxkW2RhdGEtdG9vbHNldC1ibG9ja3MtZmllbGQ9ImYwZTQzOTlhMDU5MzhjYzMxZTdlOWVkOTYxNjczZGFjIl0geyB0ZXh0LWFsaWduOiBsZWZ0O21hcmdpbi1sZWZ0OiA3JTsgfSAudGItZmllbGRbZGF0YS10b29sc2V0LWJsb2Nrcy1maWVsZD0iZjBlNDM5OWEwNTkzOGNjMzFlN2U5ZWQ5NjE2NzNkYWMiXSBhIHsgdGV4dC1kZWNvcmF0aW9uOiBub25lOyB9ICAudGItY29udGFpbmVyIC50Yi1jb250YWluZXItaW5uZXJ7d2lkdGg6MTAwJTttYXJnaW46MCBhdXRvfSAud3AtYmxvY2stdG9vbHNldC1ibG9ja3MtY29udGFpbmVyLnRiLWNvbnRhaW5lcltkYXRhLXRvb2xzZXQtYmxvY2tzLWNvbnRhaW5lcj0iZWNlNTdhMDBiYjQzM2UwZTI4YTk4MWI0MzQ3MDY5M2EiXSB7IGJvcmRlci1yYWRpdXM6IDRweDtiYWNrZ3JvdW5kOiByZ2JhKCAyMzAsIDIzMCwgMjMwLCAxICk7cGFkZGluZzogMjVweDttYXJnaW46IDQ1cHggMTAlIDI1cHggMTAlO2JvcmRlcjogMnB4IHNvbGlkIHJnYmEoIDAsIDAsIDAsIDEgKTsgfSBAbWVkaWEgb25seSBzY3JlZW4gYW5kIChtYXgtd2lkdGg6IDc5OXB4KSB7IC50Yi1jb250YWluZXIgLnRiLWNvbnRhaW5lci1pbm5lcnt3aWR0aDoxMDAlO21hcmdpbjowIGF1dG99LnRiLWZpZWxkW2RhdGEtdG9vbHNldC1ibG9ja3MtZmllbGQ9IjRhNmE5ZTQwMzE2OWY3NDEzMjc4N2QzZmI2NDM3MzJkIl0gYSB7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgfSAgLnRiLWNvbnRhaW5lciAudGItY29udGFpbmVyLWlubmVye3dpZHRoOjEwMCU7bWFyZ2luOjAgYXV0b30udGItZ3JpZCwudGItZ3JpZD4uYmxvY2stZWRpdG9yLWlubmVyLWJsb2Nrcz4uYmxvY2stZWRpdG9yLWJsb2NrLWxpc3RfX2xheW91dHtkaXNwbGF5OmdyaWQ7Z3JpZC1yb3ctZ2FwOjI1cHg7Z3JpZC1jb2x1bW4tZ2FwOjI1cHh9LnRiLWdyaWQtaXRlbXtiYWNrZ3JvdW5kOiNkMzhhMDM7cGFkZGluZzozMHB4fS50Yi1ncmlkLWNvbHVtbntmbGV4LXdyYXA6d3JhcH0udGItZ3JpZC1jb2x1bW4+Knt3aWR0aDoxMDAlfS50Yi1ncmlkLWNvbHVtbi50Yi1ncmlkLWFsaWduLXRvcHt3aWR0aDoxMDAlO2Rpc3BsYXk6ZmxleDthbGlnbi1jb250ZW50OmZsZXgtc3RhcnR9LnRiLWdyaWQtY29sdW1uLnRiLWdyaWQtYWxpZ24tY2VudGVye3dpZHRoOjEwMCU7ZGlzcGxheTpmbGV4O2FsaWduLWNvbnRlbnQ6Y2VudGVyfS50Yi1ncmlkLWNvbHVtbi50Yi1ncmlkLWFsaWduLWJvdHRvbXt3aWR0aDoxMDAlO2Rpc3BsYXk6ZmxleDthbGlnbi1jb250ZW50OmZsZXgtZW5kfSAud3AtYmxvY2stdG9vbHNldC1ibG9ja3MtZ3JpZC50Yi1ncmlkW2RhdGEtdG9vbHNldC1ibG9ja3MtZ3JpZD0iODMxYTgzMWQ1ZTNjYTYyYzJhMWM2MjVkODg2MzU4ODMiXSB7IGdyaWQtdGVtcGxhdGUtY29sdW1uczogbWlubWF4KDAsIDAuNWZyKSBtaW5tYXgoMCwgMC41ZnIpO2dyaWQtYXV0by1mbG93OiByb3cgfSAud3AtYmxvY2stdG9vbHNldC1ibG9ja3MtZ3JpZC50Yi1ncmlkW2RhdGEtdG9vbHNldC1ibG9ja3MtZ3JpZD0iODMxYTgzMWQ1ZTNjYTYyYzJhMWM2MjVkODg2MzU4ODMiXSA+IC50Yi1ncmlkLWNvbHVtbjpudGgtb2YtdHlwZSgybiArIDEpIHsgZ3JpZC1jb2x1bW46IDEgfSAud3AtYmxvY2stdG9vbHNldC1ibG9ja3MtZ3JpZC50Yi1ncmlkW2RhdGEtdG9vbHNldC1ibG9ja3MtZ3JpZD0iODMxYTgzMWQ1ZTNjYTYyYzJhMWM2MjVkODg2MzU4ODMiXSA+IC50Yi1ncmlkLWNvbHVtbjpudGgtb2YtdHlwZSgybiArIDIpIHsgZ3JpZC1jb2x1bW46IDIgfSAud3AtYmxvY2stdG9vbHNldC1ibG9ja3MtZ3JpZC1jb2x1bW4udGItZ3JpZC1jb2x1bW5bZGF0YS10b29sc2V0LWJsb2Nrcy1ncmlkLWNvbHVtbj0iMzAzNGZiZTg4NmMxMTA1NGU5NWI0NmIwOWQzZTQxMTIiXSB7IGRpc3BsYXk6IGZsZXg7IH0gLnRiLWltYWdle3Bvc2l0aW9uOnJlbGF0aXZlO3RyYW5zaXRpb246dHJhbnNmb3JtIDAuMjVzIGVhc2V9LndwLWJsb2NrLWltYWdlIC50Yi1pbWFnZS5hbGlnbmNlbnRlcnttYXJnaW4tbGVmdDphdXRvO21hcmdpbi1yaWdodDphdXRvfS50Yi1pbWFnZSBpbWd7bWF4LXdpZHRoOjEwMCU7aGVpZ2h0OmF1dG87d2lkdGg6YXV0bzt0cmFuc2l0aW9uOnRyYW5zZm9ybSAwLjI1cyBlYXNlfS50Yi1pbWFnZSAudGItaW1hZ2UtY2FwdGlvbi1maXQtdG8taW1hZ2V7ZGlzcGxheTp0YWJsZX0udGItaW1hZ2UgLnRiLWltYWdlLWNhcHRpb24tZml0LXRvLWltYWdlIC50Yi1pbWFnZS1jYXB0aW9ue2Rpc3BsYXk6dGFibGUtY2FwdGlvbjtjYXB0aW9uLXNpZGU6Ym90dG9tfS50Yi1jb250YWluZXIgLnRiLWNvbnRhaW5lci1pbm5lcnt3aWR0aDoxMDAlO21hcmdpbjowIGF1dG99IC53cC1ibG9jay10b29sc2V0LWJsb2Nrcy1jb250YWluZXIudGItY29udGFpbmVyW2RhdGEtdG9vbHNldC1ibG9ja3MtY29udGFpbmVyPSJkNGFiODcxZjE5ZDE4ZTQ3ZTdiZDc0MzJjMWQ0OGM1OSJdIHsgcGFkZGluZzogMHB4O21hcmdpbjogLTE1cHggMHB4IDBweCAwcHg7IH0gLnRiLWZpZWxkW2RhdGEtdG9vbHNldC1ibG9ja3MtZmllbGQ9ImYwZTQzOTlhMDU5MzhjYzMxZTdlOWVkOTYxNjczZGFjIl0gYSB7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgfSAgLnRiLWNvbnRhaW5lciAudGItY29udGFpbmVyLWlubmVye3dpZHRoOjEwMCU7bWFyZ2luOjAgYXV0b30gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWNvbnRhaW5lci50Yi1jb250YWluZXJbZGF0YS10b29sc2V0LWJsb2Nrcy1jb250YWluZXI9ImVjZTU3YTAwYmI0MzNlMGUyOGE5ODFiNDM0NzA2OTNhIl0geyBwYWRkaW5nOiAxMHB4O21hcmdpbjogMHB4IDBweCAyMHB4IDBweDsgfSAgfSBAbWVkaWEgb25seSBzY3JlZW4gYW5kIChtYXgtd2lkdGg6IDU5OXB4KSB7IC50Yi1jb250YWluZXIgLnRiLWNvbnRhaW5lci1pbm5lcnt3aWR0aDoxMDAlO21hcmdpbjowIGF1dG99LnRiLWZpZWxkW2RhdGEtdG9vbHNldC1ibG9ja3MtZmllbGQ9IjRhNmE5ZTQwMzE2OWY3NDEzMjc4N2QzZmI2NDM3MzJkIl0gYSB7IHRleHQtZGVjb3JhdGlvbjogbm9uZTsgfSAgLnRiLWNvbnRhaW5lciAudGItY29udGFpbmVyLWlubmVye3dpZHRoOjEwMCU7bWFyZ2luOjAgYXV0b30udGItZ3JpZCwudGItZ3JpZD4uYmxvY2stZWRpdG9yLWlubmVyLWJsb2Nrcz4uYmxvY2stZWRpdG9yLWJsb2NrLWxpc3RfX2xheW91dHtkaXNwbGF5OmdyaWQ7Z3JpZC1yb3ctZ2FwOjI1cHg7Z3JpZC1jb2x1bW4tZ2FwOjI1cHh9LnRiLWdyaWQtaXRlbXtiYWNrZ3JvdW5kOiNkMzhhMDM7cGFkZGluZzozMHB4fS50Yi1ncmlkLWNvbHVtbntmbGV4LXdyYXA6d3JhcH0udGItZ3JpZC1jb2x1bW4+Knt3aWR0aDoxMDAlfS50Yi1ncmlkLWNvbHVtbi50Yi1ncmlkLWFsaWduLXRvcHt3aWR0aDoxMDAlO2Rpc3BsYXk6ZmxleDthbGlnbi1jb250ZW50OmZsZXgtc3RhcnR9LnRiLWdyaWQtY29sdW1uLnRiLWdyaWQtYWxpZ24tY2VudGVye3dpZHRoOjEwMCU7ZGlzcGxheTpmbGV4O2FsaWduLWNvbnRlbnQ6Y2VudGVyfS50Yi1ncmlkLWNvbHVtbi50Yi1ncmlkLWFsaWduLWJvdHRvbXt3aWR0aDoxMDAlO2Rpc3BsYXk6ZmxleDthbGlnbi1jb250ZW50OmZsZXgtZW5kfSAud3AtYmxvY2stdG9vbHNldC1ibG9ja3MtZ3JpZC50Yi1ncmlkW2RhdGEtdG9vbHNldC1ibG9ja3MtZ3JpZD0iODMxYTgzMWQ1ZTNjYTYyYzJhMWM2MjVkODg2MzU4ODMiXSB7IGdyaWQtdGVtcGxhdGUtY29sdW1uczogbWlubWF4KDAsIDFmcik7Z3JpZC1hdXRvLWZsb3c6IHJvdyB9IC53cC1ibG9jay10b29sc2V0LWJsb2Nrcy1ncmlkLnRiLWdyaWRbZGF0YS10b29sc2V0LWJsb2Nrcy1ncmlkPSI4MzFhODMxZDVlM2NhNjJjMmExYzYyNWQ4ODYzNTg4MyJdICA+IC50Yi1ncmlkLWNvbHVtbjpudGgtb2YtdHlwZSgxbisxKSB7IGdyaWQtY29sdW1uOiAxIH0gLndwLWJsb2NrLXRvb2xzZXQtYmxvY2tzLWdyaWQtY29sdW1uLnRiLWdyaWQtY29sdW1uW2RhdGEtdG9vbHNldC1ibG9ja3MtZ3JpZC1jb2x1bW49IjMwMzRmYmU4ODZjMTEwNTRlOTViNDZiMDlkM2U0MTEyIl0geyBkaXNwbGF5OiBmbGV4OyB9IC50Yi1pbWFnZXtwb3NpdGlvbjpyZWxhdGl2ZTt0cmFuc2l0aW9uOnRyYW5zZm9ybSAwLjI1cyBlYXNlfS53cC1ibG9jay1pbWFnZSAudGItaW1hZ2UuYWxpZ25jZW50ZXJ7bWFyZ2luLWxlZnQ6YXV0bzttYXJnaW4tcmlnaHQ6YXV0b30udGItaW1hZ2UgaW1ne21heC13aWR0aDoxMDAlO2hlaWdodDphdXRvO3dpZHRoOmF1dG87dHJhbnNpdGlvbjp0cmFuc2Zvcm0gMC4yNXMgZWFzZX0udGItaW1hZ2UgLnRiLWltYWdlLWNhcHRpb24tZml0LXRvLWltYWdle2Rpc3BsYXk6dGFibGV9LnRiLWltYWdlIC50Yi1pbWFnZS1jYXB0aW9uLWZpdC10by1pbWFnZSAudGItaW1hZ2UtY2FwdGlvbntkaXNwbGF5OnRhYmxlLWNhcHRpb247Y2FwdGlvbi1zaWRlOmJvdHRvbX0udGItY29udGFpbmVyIC50Yi1jb250YWluZXItaW5uZXJ7d2lkdGg6MTAwJTttYXJnaW46MCBhdXRvfSAud3AtYmxvY2stdG9vbHNldC1ibG9ja3MtY29udGFpbmVyLnRiLWNvbnRhaW5lcltkYXRhLXRvb2xzZXQtYmxvY2tzLWNvbnRhaW5lcj0iZDRhYjg3MWYxOWQxOGU0N2U3YmQ3NDMyYzFkNDhjNTkiXSB7IHBhZGRpbmc6IDBweDttYXJnaW46IC0xNXB4IDBweCAwcHggMHB4OyB9IC50Yi1maWVsZFtkYXRhLXRvb2xzZXQtYmxvY2tzLWZpZWxkPSJmMGU0Mzk5YTA1OTM4Y2MzMWU3ZTllZDk2MTY3M2RhYyJdIGEgeyB0ZXh0LWRlY29yYXRpb246IG5vbmU7IH0gIC50Yi1jb250YWluZXIgLnRiLWNvbnRhaW5lci1pbm5lcnt3aWR0aDoxMDAlO21hcmdpbjowIGF1dG99IC53cC1ibG9jay10b29sc2V0LWJsb2Nrcy1jb250YWluZXIudGItY29udGFpbmVyW2RhdGEtdG9vbHNldC1ibG9ja3MtY29udGFpbmVyPSJlY2U1N2EwMGJiNDMzZTBlMjhhOTgxYjQzNDcwNjkzYSJdIHsgcGFkZGluZzogMTBweDttYXJnaW46IDBweCAwcHggMjBweCAwcHg7IH0gIH0g
🎙️ w/ @MelMitchell1 on testing LLMs and the debate over understanding.
Why …
🎙️ w/ @MelMitchell1 on testing LLMs and the debate over understanding.
Why LLMs should be probed differently than humans, OpenAI’s lack of transparency, the human drive for compression, calling for a new science of intelligence, and much more.
Currents 088: Melanie Mitchell on AI Measurement and Understanding – The Jim Rutt Show
Jim talks with Melanie Mitchell about her critique of applying standardized exams to LLMs and the debate over understanding in AI. They discuss ChatGPT and GPT-4’s performance on standardized exams, questioning the underlying assumptions, OpenAI’s lack of transparency, soon-to-be-released open-source LLMs, prompt engineering, making GPT its own skyhook to reduce hallucinations, the number of parameters in …
Tagged with :