Can I use the Bitcoin Core CLI to know the block height at which my transaction was mined?

For a transaction that's not considered "yours" by the Bitcoin Core wallet, the only way to learn anything about it is through the optional transaction index.

With the transaction index (-txindex on the command line or txindex=1 in the configuration file), you can use the getrawtransaction RPC with a txid to find the transaction in the chain. If you add a true after the txid, it'll also report the transaction in decoded format, along with other information such as the number of confirmations. E.g. you'd run:

bitcoin-cli getrawtransaction f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16 true

and get as response:

{ "in_active_chain": true, "txid": "f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16", "hash": "f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16", "version": 1, "size": 275, "vsize": 275, "weight": 1100, "locktime": 0, "vin": ..., "vout": ..., "hex": "0100000001c997a5e56e104102fa209c6a852dd90660a20b2d9c352423edce25857fcd3704000000004847304402204e45e16932b8af514961a1d3a1a25fdf3f4f7732e9d624c6c61548ab5fb8cd410220181522ec8eca07de4860a4acdd12909d831cc56cbbac4622082221a8768d1d0901ffffffff0200ca9a3b00000000434104ae1a62fe09c5f51b13905f07f06b99a2f7159b2225f374cd378d71302fa28414e7aab37397f554a7df5f142c21c1b7303b8a0626f1baded5c72a704f7e6cd84cac00286bee0000000043410411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e0eaddfb84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3ac00000000", "blockhash": "00000000d1145790a8694403d4063f323d499e655c83426834d4ce2f8dd4a2ee", "confirmations": 732767, "time": 1231731025, "blocktime": 1231731025
}

If you already know in which block the transaction was confirmed, you can verify it using the same RPC (even without transaction index):

bitcoin-cli getrawtransaction f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16 true 00000000d1145790a8694403d4063f323d499e655c83426834d4ce2f8dd4a2ee

These commands can also be issued in the Bitcoin-Qt debug console.

You can get bonuses upto $100 FREE BONUS when you:
💰 Install these recommended apps:
💲 SocialGood - 100% Crypto Back on Everyday Shopping
💲 xPortal - The DeFi For The Next Billion
💲 CryptoTab Browser - Lightweight, fast, and ready to mine!
💰 Register on these recommended exchanges:
🟡 Binance🟡 Bitfinex🟡 Bitmart🟡 Bittrex🟡 Bitget
🟡 CoinEx🟡 Crypto.com🟡 Gate.io🟡 Huobi🟡 Kucoin.