HDInsight Kafka Cluster not able to access primary storage Blob account

Multi tool use
Multi tool use


HDInsight Kafka Cluster not able to access primary storage Blob account



I spun Kafka cluster with blob storage (general purpose v1) as primary storage. No other storage attached. However kafka nodes not able to access blob files on same storage. Spark/R clusters can access files on same blob.
I created another kafka cluster but issue is still there. I have provided different container names for each cluster.



Command: hdfs dfs -ls wasb://elckafka2@demostoragepoc.blob.core.windows.net/


hdfs dfs -ls wasb://elckafka2@demostoragepoc.blob.core.windows.net/



Error:



ls: org.apache.hadoop.fs.azure.AzureException: No credentials found
for account demostoragepoc.blob.core.windows.net in the configuration,
and its container elckafka2 is not accessible using anonymous
credentials. Please check if the container exists first. If it is not
publicly available, you have to provide account credentials.



Thanks!




1 Answer
1



Note: Private containers in storage accounts that are NOT connected to
a cluster: You can't access the blobs in the containers unless you
define the storage account when you submit the WebHCat jobs.



For your understanding, I have created three containers as following;



enter image description here



If you access containers using HDInsight, you will get the same error message for private and blob public access level and gives desired output for the Container public access level.



enter image description here



For more details, refer “HDInsight Storage architecture” and “Hadoop Azure Support: Azure Blob Storage”.





I could access HDFS by modifying core-site.xml
– Hemant Chandurkar
yesterday






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

k 6gY,1 c4k4uH04RzvX 3qfvf1 8 ZdIEUqd9 wnxNFl1 nsUraMBk00LTl R,AiLRC,c
-
e3mv1ZkVd,pjks ESbSXATgUxrh dk0U7QDQcoxodQD1PCMGO9l3g Ip6 VTbvE,YX XbsCLlaAht5L

Popular posts from this blog

PHP contact form sending but not receiving emails

PHP contact form sending but not receiving emails I have created a contact form using PHP to send the data. The form seems to send fine as there are no errors and success message appears, however I am not receving the emails into the designated inbox. I am using PHPMailer as I originally tried just using the php 'mail' command which I now understand is a bit hit and miss. I cannot see why I would not be receving the emails so I would be very grateful for any help that could be given. I'm quite new to PHP so please be patient with me :) <?php use PHPMailerPHPMailerPHPMailer; use PHPMailerPHPMailerException; $msg = ""; if (isset($_POST['submit'])) { require 'phpmailer/src/Exception.php'; require 'phpmailer/src/PHPMailer.php'; require 'phpmailer/src/SMTP.php'; function sendemail ($to, $from, $body) { $mail = new PHPMailer(true); $mail->setFrom($from); $mail->addAddress($to); $ma...
Read more

Do graphics cards have individual ID by which single devices can be distinguished?

Do graphics cards have individual ID by which single devices can be distinguished? I have several graphics cards of the same manufacturer and same model and I want to distinguish them not only by the pcie socket they reside in but by some individual value I can read from the hardware. Is there such value and if so what would it be called and how could I access it? Right now I'm using modern nvidia cards (10xx series) and nvidia inspector.but I'd prefer if there was any value that was also applicable to amd. I have programming and scripting knowledge. I am running Windows 10 1 Answer 1 This is a powershell example to pull information from video cards. The DeviceID is supposed to be unique but only if the manufacturer supports Windows Display Driver Model (WDDM). (I copied this directly from docs.microsoft.com) $strComputer = "." $colItems = get-wmiobject -class "Win32_VideoC...
Read more

Create weekly swift ios local notifications

Create weekly swift ios local notifications I can create daily notifications but not weekly ones using swift 3 ios 10 User Notifications framework. Daily works fine but when I add in the .weekday parameter it doesn't send me a notification. The code is: for i in 1 ... 7 { let center = UNUserNotificationCenter.current(); let content = UNMutableNotificationContent(); content.title = "Title"; content.body = "content"; content.sound = UNNotificationSound.default(); var dateComponents = DateComponents(); dateComponents.weekday = i; // hours is 00 to 11 in string format if (daynight == "am") { dateComponents.hour = Int(hours); } else { dateComponents.hour = Int(hours)! + 12; } // mins is 00 to 59 in string format dateCom...
Read more