Okay. Thanks for your help. It’s a lot faster with TCP without UDP. Robert, could you put UDP back in for now so we can keep testing. My DevOps person keeps telling me that UDP is open, but obviously it’s not. I’m going to tell my DevOps person how to use the command line tools to test whether UDP is open or not.
Thanks,
Chris
> On Apr 7, 2016, at 5:33 PM, Bartos, Christopher <bartos.25@osu.edu> wrote:
>
> Hello,
>
> Now I can definitely confirm that my resolver is going slow even with UDP open.
>
> Here are my customized rawHandleValues functions (only the relevant ones):
>
> /*********************************************************************
> * Updated the BDBJE version of getRawHandleValues.
> * It will attempt to use BDBJE to getRawHandleValues() for this
> * server.
> * If getRawHandleValues() comes back with NULL, we know that this
> * server doesn't have the handle.
> * Next, we have to loop through all the other handle servers that
> * we have to see if the handle exists on the other servers.
> *
> * For each server URL, it will attempt to make an HTTP request and
> * follow the redirects.
> *
> * If the URL that we go to, and the URL that
> * is redirected are different, then we want to redirect to that
> * URL.
> *
> * If they are the same, we KNOW that the server doesn't have
> * that handle.
> *********************************************************************/
> public final byte[][] getRawHandleValues(byte handle[], int indexList[],
> byte typeList[][])
> throws HandleException
> {
> // getHandleServerEndpoints();
> String myHandle = net.handle.hdllib.Util.decodeString(handle);
> String suffix = myHandle.split("/")[1];
> String handleServerUrl = null;
> try {
> handleServerUrl = handleExistsOnExternalHandleServer(suffix);
> } catch (Exception e) {}
> if (handleServerUrl != null) {
> return getRawHandleValuesFromExternalHandleServer(handleServerUrl);
> } else {
> byte[][] values = bdbjeStorage.getRawHandleValues(handle, indexList, typeList);
> if (values != null) {
> return values;
> } else {
> return null;
> }
> }
> }
>
> /*********************************************************************
> * This will use the URL, and build a HandleValue. This will tell
> * Handle Server to redirect to that URL.
> **********************************************************************/
> private byte[][] getRawHandleValuesFromExternalHandleServer(String url)
> throws HandleException
> {
> try {
> HandleValue value = new HandleValue();
>
> value.setIndex(100);
> value.setType(Util.encodeString("URL"));
> value.setData(Util.encodeString(url));
> value.setTTLType((byte) 0);
> value.setTTL(100);
> value.setTimestamp(100);
> value.setReferences(null);
> value.setAdminCanRead(true);
> value.setAdminCanWrite(false);
> value.setAnyoneCanRead(true);
> value.setAnyoneCanWrite(false);
>
> List<HandleValue> values = new LinkedList<HandleValue>();
>
> values.add(value);
>
> byte[][] rawValues = new byte[values.size()][];
>
> for (int i = 0; i < values.size(); i++)
> {
> HandleValue hvalue = values.get(i);
>
> rawValues[i] = new byte[Encoder.calcStorageSize(hvalue)];
> Encoder.encodeHandleValue(rawValues[i], 0, hvalue);
> }
>
> return rawValues;
> } catch (Exception e) {
> return null;
> }
> }
>
> /*********************************************************************
> * Loops through the ArrayList of Handle Server hosts and determines
> * if it redirects to the target URL. If it does, simply return the
> * URL.
> **********************************************************************/
> public String handleExistsOnExternalHandleServer(String suffix)
> throws Exception
> {
> // for (String endpoint : this.handleServerHosts)
> // {
> // try {
> // String externalConnection = "http://pandora.lib.ohio-state.edu:8000/1811/";;
> // URL hostAddress = new URL(endpoint);
> // URL serverAddress = new URL(hostAddress + suffix);
> // String url = serverAddress.toString();
> //
> // //set up out communications stuff
> // HttpURLConnection connection = null;
> //
> // connection = (HttpURLConnection)serverAddress.openConnection();
> // connection.setInstanceFollowRedirects(false);
> // connection.setRequestMethod("GET");
> // connection.connect();
> //
> // // check the status code and make sure it's not 404
> // int statusCode = connection.getResponseCode();
> // if (statusCode == 302) {
> // return (new URL(externalConnection + suffix)).toString();
> // } else {
> // return null;
> // }
> // } catch (Exception e) {}
> // }
> // return null;
> String handle = "1811/"+suffix;
> HandleResolver resolver = new HandleResolver();
> try {
> ResolutionRequest req = new ResolutionRequest(handle.getBytes("UTF8"), null, null, null);
> AbstractResponse response = resolver.processRequest(req);
> if (response.responseCode == AbstractMessage.RC_SUCCESS) {
> return "http://kb.osu.edu/dspace/handle/"; + handle;
> } else {
> return null;
> }
> } catch (Throwable t) {
> return null;
> }
> }
>
> Thanks,
>
> Chris
>
>> On Apr 4, 2016, at 1:00 PM, Robert Tupelo-Schneck <schneck@cnri.reston.va.us> wrote:
>>
>> I still do not get responses to UDP 2642 at 140.254.87.133.
>>
>> You could check your access.log file to see if UDP requests are being received. In some cases, incoming UDP works but not outgoing. That could be a clue. (Although in your case, UDP 2641 is working at the same IP address, so I would expect the same firewall configuration to work for both.)
>>
>> If you have access to a machine outside your firewall, you can check the behavior yourself, by using the hdl-admintool to resolve (for example) 1811.1/1. In your console, you will see output containing several lines of "sending HDL-UDP request" before "sending HDL-TCP request" followed by "received HDL-TCP response". That indicates the HDL-UDP requests did not receive a response.
>>
>> You could also send me your config.dct to confirm that the server configuration doesn't have some unexpected obstacle.
>>
>> If all else fails, we can change the site information for your 1811.1 prefix to not advertise UDP capability.
>>
>> Robert
>>
>>> On 2016-04-04, at 12:46, Bartos, Christopher <bartos.25@osu.edu> wrote:
>>>
>>> UDP is now open and it still takes ~4 seconds to resolve.
>>>
>>>> On Apr 1, 2016, at 1:27 PM, Robert Tupelo-Schneck <schneck@cnri.reston.va.us> wrote:
>>>>
>>>> You need to open both UDP 2642 and TCP 2642. Currently TCP 2642 is open, but not UDP 2642.
>>>>
>>>> Robert
>>>>
>>>>> On 2016-04-01, at 13:08, Bartos, Christopher <bartos.25@osu.edu> wrote:
>>>>>
>>>>> I opened up 2642 and it seems to be the same speed as before.
>>>>>
>>>>>> On Mar 30, 2016, at 1:53 PM, Bartos, Christopher <bartos.25@osu.edu> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> My plan is to open udp and tcp on port 2642. Thanks for all your help. Sorry, Jane, for not seeing your email last week.
>>>>>>
>>>>>> Thanks everyone!
>>>>>>
>>>>>> Forgive the typo... your 1811.1 prefix advertises service on UDP 2642 (not 2641).
>>>>>>
>>>>>> Robert
>>>>>>
>>>>>>> On 2016-03-30, at 13:36, Robert Tupelo-Schneck <schneck@cnri.reston.va.us> wrote:
>>>>>>>
>>>>>>> Chris, your 1811 server is open to public resolution at UDP 2641, TCP 2641, and HTTP 8000.
>>>>>>>
>>>>>>> Your 1811.1 server is only open on HTTP 8002, but your prefix does advertise service on UDP 2641 and TCP 2642. As Scott suggested, this is why resolution is slow.
>>>>>>>
>>>>>>> Best is to open UDP 2642 and TCP 2642 at your firewall. If strictly necessary, CNRI can change your prefix information to advertise only TCP 8002.
>>>>>>>
>>>>>>> On a separate note, the resolution behavior at 1811.1 is unusual. So 1811.1/1112 resolves to the same value at 1811/1112. I'm curious to hear more about the value of that to you. Do you want 1811/1112 to be the public handle of record, as it were, or 1811.1/1112?
>>>>>>>
>>>>>>> Also, you should check your spam filter, as Jane Euler at CNRI emailed you about the matter of the ports being closed last week.
>>>>>>>
>>>>>>> Best,
>>>>>>> Robert
>>>>>>>
>>>>>>>> On 2016-03-30, at 12:46, Scott Prater <scott.prater@wisc.edu> wrote:
>>>>>>>>
>>>>>>>> Yes, that would most definitely be a cause for a delay. Can you temporarily open the handle server TCP and UDP ports to the world, as see if the resolution is faster?
>>>>>>>>
>>>>>>>> -- Scott
>>>>>>>>
>>>>>>>> On 03/30/2016 11:40 AM, Bartos, Christopher wrote:
>>>>>>>>> The only ones we have open to the world is our HTTP protocols opened on
>>>>>>>>> 8002 where the handle server is running on.
>>>>>>>>>
>>>>>>>>> We have 2 handle servers 1) DSpace instance and 2) Regular Handle Server
>>>>>>>>>
>>>>>>>>> They are running on HTTP 8000 and HTTP 8002 respectively. Neither of the
>>>>>>>>> UDP / TCP ports are open to the world. Would that be cause of slowness?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>>> On Mar 30, 2016, at 12:24 PM, Scott Prater <scott.prater@wisc.edu
>>>>>>>>>> <mailto:scott.prater@wisc.edu>> wrote:
>>>>>>>>>>
>>>>>>>>>> What ports do you have open to the world on your handle server, to
>>>>>>>>>> which protocols? In my experience, slowness in resolution is usually
>>>>>>>>>> because the udp handle port is closed to the world, causing the client
>>>>>>>>>> to fall back to tcp, and maybe even http, to resolve.
>>>>>>>>>>
>>>>>>>>>> Looking at the output of the handle client as it attempts to resolve
>>>>>>>>>> will give you a hint as to where the delay is.
>>>>>>>>>>
>>>>>>>>>> -- Scott
>>>>>>>>>>
>>>>>>>>>> On 03/30/2016 10:46 AM, Bartos, Christopher wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I've overridden the HandleStorage class in Handle Server so that it will
>>>>>>>>>>> attempt to resolve a specific prefix internally, and if it doesn’t
>>>>>>>>>>> resolve, it will attempt to resolve itself, meaning, it runs the normal
>>>>>>>>>>> BDJBEHandleStorage class to resolve it’s own handle.
>>>>>>>>>>>
>>>>>>>>>>> Here is my new class:
>>>>>>>>>>> https://github.com/lupulin/MultiServerHandleStorage/blob/master/MultiServerHandleStorage.java
>>>>>>>>>>>
>>>>>>>>>>> My question: it’s pretty quick locally. You can check it:
>>>>>>>>>>>
>>>>>>>>>>> http://140.254.87.133:8002/1811.1/OSU123 will resolve itself.
>>>>>>>>>>> http://140.254.87.133:8002/1811.1/1112 will resolve a different handle
>>>>>>>>>>> server.
>>>>>>>>>>>
>>>>>>>>>>> However, going through hdl.handle.net <http://hdl.handle.net>:
>>>>>>>>>>>
>>>>>>>>>>> http://hdl.handle.net/1811.1/OSU123andhttp://hdl.handle.net/1811.1/1112
>>>>>>>>>>>
>>>>>>>>>>> Is very, very slow.
>>>>>>>>>>>
>>>>>>>>>>> My question, is that I’m wondering if there is a way that we can speed
>>>>>>>>>>> up the HandleStorage implementation on our side or if there is nothing
>>>>>>>>>>> we can do. Also, if there is a better way to do what we are attempting.
>>>>>>>>>>>
>>>>>>>>>>> Our goal is to have 2 handle servers, both with the same prefix, but the
>>>>>>>>>>> one that is publicly available will use our custom HandleStorage class.
>>>>>>>>>>>
>>>>>>>>>>> Thanks in advance,
>>>>>>>>>>>
>>>>>>>>>>> Chris
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Handle-Info mailing list
>>>>>>>>>>> Handle-Info@cnri.reston.va.us <mailto:Handle-Info@cnri.reston.va.us>
>>>>>>>>>>> http://www.handle.net/mailman/listinfo/handle-info
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Scott Prater
>>>>>>>>>> Shared Development Group
>>>>>>>>>> General Library System
>>>>>>>>>> University of Wisconsin - Madison
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Scott Prater
>>>>>>>> Shared Development Group
>>>>>>>> General Library System
>>>>>>>> University of Wisconsin - Madison
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Handle-Info mailing list
>>>>>>>> Handle-Info@cnri.reston.va.us
>>>>>>>> http://www.handle.net/mailman/listinfo/handle-info
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Handle-Info mailing list
>>>>>> Handle-Info@cnri.reston.va.us
>>>>>> http://www.handle.net/mailman/listinfo/handle-info
>>>>>
>>>>> _______________________________________________
>>>>> Handle-Info mailing list
>>>>> Handle-Info@cnri.reston.va.us
>>>>> http://www.handle.net/mailman/listinfo/handle-info
>>>>
>>>
>>
>
Attachment:
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Handle-Info mailing list Handle-Info@cnri.reston.va.us http://www.handle.net/mailman/listinfo/handle-info