Kaeya's problems
To begin the quest, find Kaeya at the Knights of Favonius headquarters. Talk to him with start the Secret pirate treasure questline. He will send you to the Adventurers Guild to collect testimonies from nearby adventurers. Talk to Cyrus, who is standing in front of the Adventurers Guild building, then you will be sent back to Kaeya. The return to Kaeya will complete the Kaeya's problems quest and start the Mystery of the Arcadian ruins quest.
Meet the informant
Kaeya will send you to meet her informant outside Monstadt at Whispering woods. When you introduce yourself you will find the Informant attacked by a group of Hilichurls. Treat them and talk to the informant. She will give you a picture with a riddle on it that will guide you to the first treasure chest. You can return to Kaeya at HQ of the Knights of Favonius for a clue, but you can just keep reading to find the exact location of the chest.
Monstadt windmill
The first clue for the Mystery of the Arcadian Ruins quest reads as follows.
This index refers to the three windmills in Monstadt. Ride on the Mondstadt southwest windmill. If you can't find a path, simply climb a nearby building and slide to your destination. There is a chest waiting for you at the top. Inside you'll find another card with another clue. Your treasure hunt is not yet over.
Springvale Waterfall
The second clue takes you somewhere a little further, but it's still not too far out of reach. The index reads as follows.
You can return to Kaeya at the Knights of Favonius headquarters for a clue. It'll tell you to head to Springvale, an area south of Monstadt, and this will help narrow your search. The map with the clue should point you clearly enough to your destination, but if you can't find it, it's near a body of water under the word "Springvale" on your map. There is a waterfall there, and next to the waterfall you will find three small pillars with ice symbols on them. You'll need to freeze them with Kaeya's Cryo abilities to summon a chest containing your final clue.
Falcon Coast
The third and final clue will mark your destination on your map, so you won't have to do any detective work to figure out where you need to go. Head to the Falcon Coast and you'll find a dungeon in a cave near the beach. The recommended level is 10, so make sure your party members are strong enough before venturing inside.
It's a pretty standard dungeon, but you'll be able to use Kaeya's Cryo abilities multiple times to disable fire traps and defeat fire enemies. There is a boss battle at the end of the dungeon, but it's nothing you can't handle. After you land the final blow, the quest will be complete and you will have completed the first part of the Secret Pirate Treasure questline. There's more to come later, but for now you're done.