How I Automated Getting Webpage Titles from a List of URLs

A while ago, I had a big list of website links saved in an Excel file—over 700 of them! I needed to find out the title of each website. Doing it by hand would mean opening every link, copying the title, and pasting it back into Excel. That would take forever, and I’d probably make mistakes along the way.

The Problem

When you have a lot of data, doing simple, repetitive work by hand is really slow and easy to mess up. It’s boring and you can easily skip a link or copy something wrong.

My Solution: Using Python

So I decided to let my computer do the work for me. Python is a programming language that makes things like this much easier. I wrote a short script that reads all the links from my Excel file, visits each website on its own, grabs the title, and then saves everything back into Excel.

Here’s the script I used:

import pandas as pd

import requests

from bs4 import BeautifulSoup

# Upload your file first in Colab

df = pd.read_excel("/content/AskHareesh/AskHareeshExcel.xlsx")   # assumes your URLs are in a column named 'URL'

titles = []

for url in df['URL']:

    try:

        response = requests.get(url, timeout=10)

        soup = BeautifulSoup(response.text, "html.parser")

        title = soup.title.string if soup.title else "No Title Found"

    except Exception as e:

        title = f"Error: {e}"

    titles.append(title)

df['Title'] = titles

df.to_excel("/content/AskHareesh/AskHareeshExcelTitle.xlsx", index=False)

How to Run the Script (Even If You Don’t Have Python Installed)

You don’t need to install anything on your computer. There’s a website called Google Colab where you can run Python scripts for free:

1. Go to Google Colab (just search for it online).
2. Upload your Excel file that has the URLs.
3. Copy and paste the script above into a cell in the notebook.
4. Run the code. It will go through all your links and get the webpage titles.
5. You can then download your updated Excel file with an extra column for the page titles.

Automation can really make life easier!

Read More »

Resolving Key Conversion Error in Oracle Cloud OIC Integration

When running an Oracle Integration Cloud (OIC) integration, you may encounter the following error despite successfully testing and validating the connection:

 <![CDATA[CASDK-0041: An error occurred while invoking the REST endpoint.[[CloudInvocationException [ java.lang.ClassCastException: java.lang.ClassCastException: class org.bouncycastle.asn1.x509.SubjectPublicKeyInfo cannot be cast to class org.bouncycastle.openssl.PEMKeyPair (org.bouncycastle.asn1.x509.SubjectPublicKeyInfo and org.bouncycastle.openssl.PEMKeyPair are in unnamed module of loader 'app') ]]].The 500 Internal Server Error is a very general HTTP status code that means something has gone wrong on the server side, but the target service could not be more specific on what the exact problem is. You can trace the cURL representation of the request sent to the target service from the Oracle Integration Cloud server logs. Try invoking the target service using cURL. If the problem persists, contact the target service admin.]]>

Cause of the Error

This error occurs due to a mismatch in the private key format. The key must be in PEM format for proper authentication.

Solution

Converting the private key to the correct PEM format will resolve this issue. Use the following command to perform the conversion:

ssh-keygen -p -f <KEY> -N '' -t rsa -m pem

Replace <KEY> with the actual private key file name.

Conclusion

If you face this issue in Oracle Cloud OIC, ensure your private key is in the correct PEM format. This simple conversion step can save valuable troubleshooting time and ensure smooth integration execution.

Read More »

Trigger, Invoke, and Trigger & Invoke in OIC REST Connections

Trigger, Invoke, and Trigger & Invoke in OIC REST Connections

In Oracle Integration Cloud (OIC), REST connections allow systems to communicate. You can configure a connection in one of three ways: Trigger, Invoke, or Trigger and Invoke.

Trigger: OIC acts as a server, exposing a REST API so that external systems can receive data. Example: Third-party service receiving data using OIC Rest API.

Invoke: OIC acts as a client, calling external REST services. Example: Fetching data from an external API like a CRM system.

Trigger and Invoke: OIC can both receive and send requests using the same connection, enabling two-way communication. Example: Receiving customer data and validating it via an external service in a single flow.

By understanding these roles, you can set up flexible and efficient integrations tailored to your needs. 

Read More »

How to Create a Sample REST API in Oracle Integration Cloud (OIC)

Let's create a sample REST API using OIC with below steps

Step1: Create REST connection

Step2: Create Integration by using REST connection

Step3: Configure parameters if any and response as well

Step4: Map the elements for response if required and validate

Step5: Activate the Integration and test

We need a REST connection to create REST API using OIC.

Navigate to Design --> Connections.

Create the connection using create button.

Select REST from the list of the adapters exists

Provide the Integration Name and Integration Identifier, click on create to proceed to next steps.

Select the security policy as Basic Authentication.

Test the connection.

Save the connection.

Go to Integrations.

Click on create button to create the integration.

Select Application to create REST API

Provide the Integration Name and Integration Identifier and click on Create button.

Select the REST connection that we created earlier and give the endpoint name and description.

Click and continue button and configure the REST trigger by providing the end point URI (/{name}) so that we can pass name as part of URL and consume it and select the "Configure this endpoint to receive the response" to send the response upon calling REST API.

Click on Continue button and no need to configure the trigger parameters in this screen.

Click on continue and click on inline to add the sample JSON format for response.

Click on continue

Click on Continue

Click on Finish

Open the mapper in Edit mode and create a target node for message element in response wrapper.

Enable the developer view.

Add the below concat function and drag and drop the name from the left pane.

concat("Hi ",/nstrgmpr:execute/nstrgmpr:TemplateParameters/ns14:name,", How are you?")

Validate the mapper.

Go back to the integration after validating the mapper.

Go to the business identifiers.

Drag and drop the name as business identifier and save the integration.

Go back to all integrations after saving the integration to active the integration.

 

Enable the debug option to verify the logs and click on activate.

Refresh to check the status of the integration.

Run the integration to test REST API.

Pass the value in name and run.

Observe the response of the API.

To test the API in other tools, click on the Endpoint metadata to get the REST API URL.

Read More »

Browser Navigation issue in OAF

Usually this issue occurs when closing the popup window and moving to the next set of data of search results using table navigation bar.

Lets look at the snaps of the issue for reference.

Code which has issue:

String errorPage = "/ah/oracle/apps/aoc/AHOMUI/webui/AHOMErrorsPG&banner=home&addBreadCrumb=Y&ErrorId={@ErrId}";

String destinationURL = APPS_HTML_DIRECTORY + OAWebBeanConstants.APPLICATION_JSP + "?" + OAWebBeanConstants.JRAD_PAGE_URL_CONSTANT + "=" + errorPage;

OALinkBean errorsAdvLink = (OALinkBean)advResultsTabBean.findIndexedChildRecursive("ErrorAdv");

OABoundValueEmbedURL errorAdvJsWindow = new OABoundValueEmbedURL(errorsAdvLink, 

                                                                         "javascript:var win = window.open( '", 

                                                                         destinationURL, 

                                                                          "' , 'win', 'height=600,width=1000,left=80,top=250,status=yes,toolbar=no,menubar=no,location=no,scrollbars=yes,resizable=yes'); win.focus();");

errorsAdvLink.setAttributeValue(oracle.cabo.ui.UIConstants.ON_CLICK_ATTR, errorAdvJsWindow);

To resolve this issue, we need to retain the application module in the popup page.

Solved code:

String errorPage = "/ah/oracle/apps/aoc/AHOMUI/webui/AHOMErrorsPG&banner=home&retainAM=Y&addBreadCrumb=Y&ErrorId={@ErrId}";

String destinationURL = APPS_HTML_DIRECTORY + OAWebBeanConstants.APPLICATION_JSP + "?" + OAWebBeanConstants.JRAD_PAGE_URL_CONSTANT + "=" + errorPage;

OALinkBean errorsAdvLink = (OALinkBean)advResultsTabBean.findIndexedChildRecursive("ErrorAdv");

OABoundValueEmbedURL errorAdvJsWindow = new OABoundValueEmbedURL(errorsAdvLink, 

                                                                         "javascript:var win = window.open( '", 

                                                                         destinationURL, 

                                                                          "' , 'win', 'height=600,width=1000,left=80,top=250,status=yes,toolbar=no,menubar=no,location=no,scrollbars=yes,resizable=yes'); win.focus();");

errorsAdvLink.setAttributeValue(oracle.cabo.ui.UIConstants.ON_CLICK_ATTR, errorAdvJsWindow);

Read More »

Assign SQL value to variable in shell script

Requirement is to get the emails list from value set and send an email using shell script.

Sample Shell Script:

#!/bin/ksh

#*******************************************************************************

# Name : HH_TEST_SHELL

# Date : 19-MAY-2023

# By : AskHareesh

# Description : Execute JAR File

#*******************************************************************************

MAILBODY="Hi Team,

Please find the attached report."

MAIL_SUBJECT="Sales Order Status Report"

LOGDIR="/test/dir/"

LOGFILE="SalesOrderReport.csv"

INSTANCENAME="DEV"

VALIDATEEMAILS=`sqlplus -s $FCP_LOGIN <<-BBB

set pages 0

set feedback 0

select LISTAGG(ffv.FLEX_VALUE, ',')

from  apps.FND_FLEX_VALUE_SETS ffvs, apps.fnd_flex_values ffv

where 1=1

and ffvs.flex_value_set_id = ffv.flex_value_set_id 

and ffvs.flex_value_set_name='ASK_HAREESH_TEST'

and NVL(ffv.enabled_flag,'N')='Y'

and TRUNC(sysdate) between TRUNC(NVL(ffv.start_date_active,SYSDATE)) AND TRUNC(NVL(ffv.end_date_active,SYSDATE))

/

  EXIT

BBB

`

(echo  "$MAILBODY") | mailx -s "$INSTANCENAME : $MAIL_SUBJECT" $VALIDATEEMAILS

Read More »

Send an email with out attachment from shell script

Requirement is to send an email with out attachment using shell script.

Sample Shell Script:

#!/bin/ksh

#*******************************************************************************

# Name : HH_TEST_SHELL

# Date : 19-MAY-2023

# By : AskHareesh

# Description : Execute JAR File

#*******************************************************************************

MAILBODY="Hi Team,

Please find the attached report."

MAIL_SUBJECT="Sales Order Status Report"

VALIDATEEMAILS="askhareesh@gmail.com"

LOGDIR="/test/dir/"

LOGFILE="SalesOrderReport.csv"

INSTANCENAME="DEV"

(echo  "$MAILBODY") | mailx -s "$INSTANCENAME : $MAIL_SUBJECT" $VALIDATEEMAILS

Read More »